Andalas Journal of Electrical and Electronic Engineering Estimation of the Shoulder Joint Angle Using Brainwaves

This paper presents the angle of the shoulder joint as basic research for developing a machine interface using EEG. The raw EEG voltage signals and power density spectrum of the voltage value were used as the learning feature. Hebbian learning was used on a multilayer perceptron network for pattern classification for the estimation of joint angles 0 o , 90 o and 180 o of the shoulder joint. Experimental results showed that it was possible to correctly classify up to 63.3% of motion using voltage values of the raw EEG signal with the neural network. Further, with selected electrodes and power density spectrum features, accuracy rose to 93.3% with more stable motion estimation.


INTRODUCTION
In the recent past, Japan and indeed the world, have witnessed a declining birthrate and aging of their population. This has caused an increase in the number of older adults in the country. In such a social situation, the burden on social welfare for the aged and the ill is heightened. It is highly desired to reduce the load on the caregiver by developing systems that support the independence of the care recipients. This system promotes the independence of the individuals and thereby improves the quality of life.
Devices that seek to increase, maintain and or improve the functional capability of the user are considered assistive technologies. In cases of disability or senility, the range of equipment's range from special communication devices, mobility devices like wheelchairs, hearing aids, control interfaces amongst others. Focusing on controlled devices, control schemes vary with different methodologies utilized in various research undertakings. Modalities like joystick control, input buttons, bio-signals, gesture recognition amongst others have been used. In this research, the primary control mechanism has been on biosignals owing to its ease of recording from different human body aspects. Among the biological signals, the approach using electroencephalography (EEG), electrooculography (EOG) and electromyography (EMG) methods are employed since they are less invasive with a considerable cost-effective measurement system. EMG has been used by different authors in applications like ergonomics, prosthesis, robot controls, smart homes, muscle rehabilitation amongst others [1]- [4]. Similarly, EOG research targeting areas like robot control, human-machine interface, game control schemes, affective computing amongst others have been developed. EEG has found usage in locked-in patients who have lost motor control. Authors [5] and [6] used EEG to communicate with patients using speller tasks. Other braincomputer interfaces have been applied in varying fields [7]. Commercially, Research Laboratories have been developing a system that operate equipment using EEG signal, for example, moving electric beds and wheelchairs by measuring subtle changes in blood flow in the brain and sends the signal to electronic devices via a network [8], [9]. The Army Combat Capabilities Development Command's Army Research Laboratory has also shown interest in the usage of EEG towards cognition and ergonomics of military operations [10], [11]. Through this and other endeavors, a man-machine interface that uses a biomedical signal as an input is being actively developed. This research is focused on the EEG, which is said to be the highest order among biological signals to estimate the state of human shoulder movements. In other EEG studies, neural networks may be used for mental task classification and epileptic wave detection, but in this study, they are used for motion estimation. EEG contains a lot of information and is useful for the communication of elderly people and people with physical disabilities who place importance on intuition and low burden. Moreover, since the brain is the center of the nervous system, its function is likely to be maintained even in diverse age and illnessrelated complications. As such, there are many cases where EEG can be used even if the person becomes paralyzed or suffer from a stroke.
To record EEG, the setup involved the measurement of the surface potential of 14 lead-out sites on the scalp. The proposed system captures EEG signals emanating from arm motion to estimate shoulder movements using a neural network. As a preprocessing of the estimation, Fourier transformation is performed to make the characteristics of the EEG signal of each motion more conspicuous, considering changes in the physical condition of the subject and the measurement environment. The performance is experimentally evaluated using test subjects with a custom-built robot for visual feedback. Additionally, optimal electrode position is investigated to improve the accuracy of the proposal.
The rest of the document is organized as follows; Section 2 describes the methods applied where, EEG system setup, Fourier transformation and its usage in EEG feature extraction, as well as neural network training schemes, are described. In section 3, the results of the experiment are reported. In the section, motion estimation results and discussion are given for varying experimental settings. The paper ends with a conclusion that summarizes the findings and outlines the challenges and recommendations for further work.

Electroencephalography (EEG) measurement
In literature, the brain is subdivided into four major lobes, Frontal, Parietal, Occipital, and Temporal lobes, which are tied to different bodily functions. The frontal lobe is generally mandated with higher executive functions including emotional regulation, planning, and problem-solving. Additionally, the region contains the primary motor cortex, the major region responsible for the voluntary movement of different body parts. The parietal lobe is largely responsible for integrating somatosensory information (touch, temperature, pain etc.). It also plays a part in coordinating hand-eye motions. The Temporal lobe on the other hand contains regions dedicated to processing the sensory information (hearing, recognizing language, etc.). Some regions of the temporal lobe also assist in making sense of complex visual information (in face recognition and scenes for example). Finally, the occipital lobe is considered the major visual processing centre of the brain. Located at the back portion of the brain, its role is interpreting visual stimuli and information. By recording brain activity, it is possible to discriminate which action the brain is executing. One such approach is the use of electroencephalography to record brain waves.
The electrical activity emanating from the spontaneous potential of the brain is referred to as EEG, a short form of Electroencephalogram. It is mainly recorded by electrodes placed on the scalp, sphenoid floor, eardrum, surface of the brain, deep brain, etc. Since EEG signals contain all physiological signals, albeit, at millivolt level at best, there is a possibility that complex motion can be identified with careful signal processing and analysis.
In this research, we used Kansei Spectrum Analysis System to investigate the waveform of EEG. EEG signal was recorded during arm motion, and the raw EEG is passed through Fourier transform for feature extraction. The extracted features were then fed to the multilayer perceptron (MLP) for classification. This is as shown in Fig. 1. Fig. 2 shows the measurement equipment used and the electrode position configurations for data acquisition. Fig. 2(a) is the processor box for hardware digital filter processing, it processes EEG and provides a link to the PC for data import. Analog/Digital (A/D) conversion and amplification were performed on the raw signal by the EEG acquisition unit shown in Fig. 2(b). This device has 14 input channels and outputs the converted signal to the processing unit. For measurement, the electrodes were mounted at 14 locations as shown in Fig. 2(c) based on the international standard 10-20 method. All measurement methods were based on the right earlobe reference electrode lead. For repeatable positioning, the head cap is shown in Fig. 2(d) was used. The electrodes in use were dry leads, paste-less electrodes in the helmet. Table 1. below list the 14 EEG signals. The position for each electrode is represented as a combination of letters and numbers. The letter, in this case, represents the particular lobe i.e. Fp, Prefrontal lobe; F, Frontal lobe; T, Temporal lobe; C, Central lobe, P, Parietal lobe, and O, Occipital lobe. The numbers represent the skulls hemispherical location: Z, denotes the cerebral midline, even numbers represent the right hemisphere while odd numbers represent the left hemisphere.

Experimental setup
The experiment was conducted with test subjects seated comfortably in a chair with minimal movement to avoid motion artifacts. A baseline EEG was recorded with a rested arm, denoted in this article as the zero degrees (0 o -0 o ), for 5 seconds. After a rest period of about 5 seconds to allow for saving the raw EEG file, the user proceeded with arm motion. The first motion involved performing a front raise, hold for a second before bringing it down. This is referred to as 0 o -90° in this paper. The last motion was raising a hand to the highest possible position, holding and finally lowering. This is referred to as 0 o -180° in the remaining part of the paper. In total, the experiment recorded EEG when the user performs three actions of the shoulder joint angle of the left arm; zero degrees (0°), zero degrees to ninety degrees (0°-90°), and zero degrees to one hundred and eighty degrees (0° -180°). The operation was repeated 10 times for each action, each lasting 5 seconds. Fig. 3 illustrates the experimental setup of the EEG based arm motion angle estimation.

MOTION ESTIMATION
The target of the research is motion estimation of the shoulder joint from the EEG signal. To this end, the raw EEG signal was passed through a preprocessing step for feature extraction. Several feature selection methods have been proposed in the literature; wavelet transforms, eigenvectors, time-frequency distributions, autoregressive models to name a few [12]- [14]. In this research, the two methods of feature extraction are investigated, one involving the raw EEG data and the other involving Fourier transformed EEG data. In Fourier transform feature extraction, the characteristics of the raw EEG signal to be analyzed is derived from power spectral density (PSD) estimation. This conversion was necessary to selectively represent the EEG sampled signal as well as reduce data dimensions [15], [16].

Power spectral density
When analyzing signal waveforms such as brain waves observed from the scalp due to the activity of many cells, it is often effective to evaluate the signal waveforms separately for each frequency component. This way, the signal was discretized at fixed time intervals (∆t). This is the so-called time discretization with sampling time, ∆t. Which is also expressed as 1/∆t, the sampling frequency.
After the discrete Fourier transform of the time series signal x(n∆t) captured as a digital signal through the A/D converter to obtain X(n∆f), the power spectrum S(n∆f ) is calculated from the expected value of the square of the amplitude for each frequency. When 1/T ≡ ∆f, the frequencies are discrete, i.e. ∆f, 2∆f, … The discrete Fourier transform of x(m∆t) is given by eq (1).
is a complex quantity, but the absolute value of the complex number is the sum of the squared value of the real part and the squared value of the imaginary part and is expressed as eq (2).
Therefore, the power spectral density S(n∆f) is defined by eq (3) as shown.
Here ⟨| ( )| 2 ⟩ represents the average over several time series of length T. S(n∆f ) is the density of the power spectrum. The reason for the density is that when the sum is added from f=n∆f to m∆f, it becomes the root mean square of the signals in that frequency range. The calculated power spectral density is fed to the neural network for training.

Multilayer perceptron
In analyzing EEG signals, artificial neural network models with different architectures have been employed in the literature. Support vector machines, radial basis function, adaptive neuralfuzzy, recurrent neural network, among others have been proposed with different amounts of processing time as well as accuracy [17]- [19]. In the paper, multilayer perceptron (MLP) architecture was applied as the neural network of choice.
A perceptron is a linear binary classifier used in supervised learning used to classify a given input data. It comprises of an input, weight, threshold, summer, and an activation function. A perceptron is a machine that outputs 1 when the input is a pattern P+ which exceeds the threshold and 0 when the input is a pattern P-which is less than the threshold. Alternatively, considering that there is only one output unit, the perceptron can be said to be a machine that divides the pattern appearing in the input layer into 1 and 0. Considering Fig. 4(a), the summed input to the unit uj is expressed as Where wj correspond to the interconnection weight between j-th input and the neuron, xi is the input pattern and hj is the threshold. This way, feedforward pass is achieved. The output yi is derived as shown in eq (5) by squashing the net input uj using the Heaviside step function. The output value is converted into an output of 1 if the activation value is greater than or equal to the threshold, otherwise it is 0 as shown parametrically in equation 5.
The pictorial representation of the perceptron is as shown in Fig  4(a).  Multilayer perceptron derives its name from having a layered hierarchical network model comprising of a fully connected perceptron. Proposed by Rosenblatt in 1958, the perceptron is a model that uses the Hebbian learning rule using MacCullock-Pitts formal neurons and can acquire simple cognitive ability. In this paper, a 3-layer NN was applied as shown in Fig. 4(b) having sensory layer(S), associative layer(A), and response layer(R). In other notation, the layers are described as the input layer, hidden layer, and output layer.
In this work, fixed connections were employed between the input nodes and the hidden units whereas interconnection weights were used between the hidden units and the output neurons.

Perceptron learning
Training multilayer perceptron is different from other multilayer architectures where techniques such as backpropagation are employed. Primarily, perceptron learning is expressed as a change in the weights from the hidden layer (association layer) to the output layer, whereas the weights from the input layer to the hidden layer are not considered.
The Hebbian learning algorithm is used where learning of the perceptron is expressed as a change in the coupling coefficient(weights) from the association layer to the reaction layer. During training, various input patterns are supplied as a teaching pattern in a supervised learning model. As these patterns are being propagated forward, the error arising from the difference between the output of any node i and the target value multiplied by the input to the neuron is proportional to the change required to make the pre-synapse and the post-synapse signals equal. Equation (6) shows the general learning rule (delta rule) of the MLP. In the expression, is the learning rate, difference between the output of node and the actual training value (i.e. error) while is the input signal for the corresponding weight. It has enhanced the speed of convergence in that updating of the Hebbian synapse occurs at the same time as the occurrence of the difference between the presynaptic signal and the target value.
In vector form, the expression can be converted to a recurrence formula as in (8). (w1, w2, ..., wN) are the weight at the end of the n_th learning.

CLASSIFICATION OF PATTERNS USING MLP
Consider, an MLP network having two inputs and one output. The input to the networks is two sets of patterns P+ and P-. Such a network can be represented by a two-dimensional plane which is sometimes called the input space. Learning in a two-dimensional space involves finding the discriminant line that divides the space into subspaces occupied by the different patterns as shown in Fig.  5 below. cos w x w x w x w x      (9) Where the angle is the angle between the two vectors and can be expressed as.
cos wx wx    (10) The sign of the left-hand side of equation 7 depends on angle . The range in which is positive is between −π/2< <π/2 in radians. All the vectors in the shaded area in Fig 6 are normal to the weight vector and this line distinguishes between the groups, each having a different sense(sign) of the coupling elements of the elements. Within this range, for example, the dot product is positive, otherwise, the product will be negative. Graphically, consider three inputs x1, x2, and x3 in P+ and a randomly initialized weight vector w0 as shown in Fig. 7(a). The boundary is the line perpendicular to the weight vector (shown in blue). From the figure, vector x1 is incorrectly classified. In the next learning iteration, the new weighting vector is w1 = w0+x1. This is accompanied by a new boundary that classifies x3 incorrectly as shown in Fig. 7(b). The next learning cycle aims to adjust the weight vector and thereby the boundary to have w3 in the correct subset. This is shown in Fig. 7(c) and is followed by vector x1 on the wrong side. In the final cycle shown in Fig. 7(d), w3 = w2+x1 changes the boundary such all the vectors are on the same side of the boundary which marks the end of the training.   From the results, the waveforms hardly changed for the 0°-0°. When arm motion was initiated, there is a significant increase in power for all the channels. Further, it was confirmed that the total potential for 0°-180° was larger than 0°-90°). From literature, the area in the brain occupied by shoulder movements is narrow, with motor effects appearing in a narrow range at the parietal region centering on the electrode positions of C3, C4, F3, F4, P3, and P4 [20]- [22]. However, during the experiment, electrodes other than the frontal and parietal regions often showed significant changes as well. This is attributed to other brain functions associated with motion like motion coordination and visual feedback. As mentioned in section 2 above, the occipital region of the brain is critical in visual information processing. As the experiment is conducted, visual feedback as well as hand-eye coordination is required, soliciting EEG from at least all other brain regions.
The location of the action potential at 0°-90° and 0°-180° may change each time since EEG is stochastic in nature. The next subsection addresses this challenge by utilizing a machine learning model to predict motion.

Motion estimation
In motion estimation, the focus is on pattern recognition to discriminate active arm motion to overcome the problem of the stochastic nature of EEG in determining action potential locations. All 14 electrodes were used for learning and the performance was compared with and without feature extraction. At this time, a neural network was used as a pattern recognition method for discrimination. Additionally, optimal channels with clear discrimination were evaluated to reduce the number of channels needed.

Angle estimation with raw EEG signal
In this experiment, the raw EEG signal from the 14 electrodes was used for discrimination. For comparison and performance analysis, different neural network configurations were evaluated for optimal architecture. The number of input neurons was varied from 10 to 600 and the hidden neurons varied from 5 to 50 with each trial being repeated 10 times.   From the results, it can be observed that the best performance, with better consistency, resulted with 3 hidden neurons with 70%, 60% and 50% for 0°-0°, 0°-90° and 0°-180° motion which average to 60% accuracy for the entire motion range.

Angle estimation with Fast Fourier Transform feature extraction
To obtain more prominent features for each motion, FFT of the raw EEG signal was performed before training. Fig. 14 below shows an example of the FFT of the Fp1 EEG signal for 0 o -90 o and 0 o -180 o motions. From the figure, it can be seen that the spectrum envelope of the two movements is the same, however, differences are present which will make it possible to discriminate between the two movements.

Selection of electrodes
Considering that different channels had different responses each time depending on the physical condition of the subject and other external factors, a preprocessing phase for motion estimation using a neural network was introduced. Preprocessing involved selecting a few channels with the largest instantaneous electrical potential for training the network.
Taking Fig. 16 as an example, the electrode that reacts most during the experiment is O1, followed by O2 and F7 in that order. Choosing these three signals for discrimination, training was carried considering signal strengths before and after the reaction as the characteristic quantities for discrimination. Figure 16. Selection of Electrodes The performance of the angle classifier without and with channel selection and feature extraction using FFT power spectrum is as shown in Fig. 17 and Fig. 18, respectively. This corresponds to an overall classification accuracy of 93% for the overall system. The improvement is attributed to the exclusion of signals with minimal or no variations as part of the training data.

CONCLUSIONS
This paper presented joint angle estimation by classifying EEG signals corresponding to three joint angles of the shoulder joint: 0 o -0 o , 0 o -90 o and 0 o -180 o . Multilayer perceptron neural network was trained using Hebbian learning to classify the electrode signals. Experiments involved the development of the algorithm and the determination of the optimal network architecture in terms of the neuron in the input and the hidden layer. Further, the performance of the classification system operating on raw EEG data was compared to the performance with feature extraction using FFT power spectrum with and without preprocessing. From the results, the optimal number of input neurons for a system trained with raw EEG signals was 42 neurons and 3 neurons in the hidden layer. Training with feature extraction yielded better results with the same number of hidden neurons albeit having 30fold the number of input neurons. The performance of the classifier with channel selection outweighed the others that involved all the channels.