Modification of body schema by use of extra robotic thumb
© The Author(s) 2018
Received: 22 September 2017
Accepted: 6 January 2018
Published: 15 January 2018
In recent years, there has been great interest in the possibility of using artificial limbs as an extension of the human body as well as replacement of lost limbs. In this paper, we develop a sixth finger system as an extension of the human body. We then investigate how an extra robotic thumb, that works as a sixth finger and gives somatosensory feedback to the user, modifies the body schema, and also affecting the self-perception of existing limbs. The sixth robotic finger is controlled with the thumb of the opposite hand, and contact information is conveyed via electrostimulation to the tip of the thumb controlling the movement. We conducted reaching task experiments with and without visual information to evaluate the level of embodiment of the sixth robotic finger and the modification of the self-perception of the finger controlling the system. The experimental results indicate that not only the sixth finger is incorporated into the body schema of the user, but also the body schema of the controlling finger is modified; ability of the brain to adapt to different scenarios and geometries of the body is also implied.
Artificial limbs had been used for long time as prostheses, and it has been studied how much are these artificial limbs embodied to the users self-perception of the body . It has been shown that an increase in embodiment improves the performance and comfortability of these artificial limbs . In recent years, great interest has arisen on the possible use of artificial limbs not just as replacement of lost limbs, but also as an extension of the human body. Some research groups have studied the use of artificial limbs as extra limbs, and their possible applications [3, 4].
Extra limbs offer the possibility to increase workspace, dexterity, strength and reduce fatigue of the user . However, extra limbs require new control strategies because extra limbs perform actions that the users have not experienced. Control strategies of extra limbs proposed previously can be divided into two approaches. First approach is the indirect control from the user by moving the artificial limbs by synergy with other movements from the user  or predicting the intended movement . Second approach is the direct control from the user mimicking the motion from a similar limb .
It has been shown that artificial limbs can be incorporated in the body schema of users when replacing lost limbs as prostheses , or as supernumerary limbs . Specially, the use of somatosensory feedback increases the sense of embodiment  and performance on grasping motions . This embodiment means that the user can have an accurate idea of the position of the artificial limb without visual information.
One example of these possible extra limbs are supernumerary fingers . That have been studied to increase grasping range and ability to perform activities with one hand . They have been also studied to help the recovery of stroke patients with somatosensory feedback  and without it .
Although there are several approaches to control devices of extra limbs, it has not been elucidated how these different control strategies affect the embodiment of the artificial limbs, or modify the body schema of the users . In this paper, we focus on the effect of controlling a supernumerary limb with direct control from the motion of a similar limb with somatosensory feedback, and its effect on the body schema of the users.
We develop a sixth finger device system as an extension of the human body, a thumb motion capture device to control movements of the device, and a electrical stimulation device to convey contact information. We then perform experiments using the sixth finger device with somatosensory feedback during 1 month with varying rest time between sessions, with and without visual feedback. In addition, the sixth finger device is controlled by mirroring the movement of the thumb from the opposite hand, while giving somatosensory feedback from contact information via electrostimulations. From the experimental results, the performance on a reaching task and the modification of self-perception of the controlling finger after each session are evaluated. Finally, we discuss about ability of the brain to adapt to different scenarios and geometries of the body though the evaluations.
Body schema was firstly put forward by Holmes and Head in 1911 . Humans can perceive intuitively the position and movement of our own body without looking because experience about our own body has been accumulated as a mental model over time. The body schema is the map of our own body generated based on the previous experience. When we move or perceive our posture, the body schema is referenced unconsciously. The body schema has various properties . Adaptability is one of these properties of the body schema. For example, when we use tools, the body schema will be changed and take the tools into consideration . We can use tools dexterously owing to this property. When we stop the use of these tools, the body schema will be restored to its initial state. We call embodiment when the prolonged use of these tools writes them strongly in the body schema.
Supramodal representation is also one of the important properties of the body schema. The body schema integrates a variety of sensory feedbacks, usually called afferent input, such as position sense, tactile sense obtained from peripheral nerves. This property makes us possible to identify the position stimulated without looking at it directly. Moreover, the body schema generates an efferent copy when we perform an activity . Efferent copy is an expected image generated before we move. We can expect the result of the movement before we actually move. The body schema associates afferent copy with efferent copy and the difference is used to update the body schema. The difference between the performed motion and the expected motion is inversely proportional to the accuracy of the efferent copy and body schema.
Somatosensory feedback has been shown to be an important factor in the performance of different interfaces  and artificial limbs . In this study, we provide pseudo-tactile feedback using electrical stimulation. The use of electrical stimulation has been shown to simulate contact and tactile feedback . This pseudo afferent input increases the extra robotic thumb embodiment. As a result, a more accurate efferent copy will be generated, and operability of the extra robotic thumb will be improved.
Extra robotic thumb system
Extra robotic thumb (ERT)
Figure 1 depicts a new developed extra robotic thumb (ERT) that is a super numerary limb device used to simulate an extra finger on the hand. The ERT attaches on the ulnar side of the palm to produce the feeling of having a second thumb on the hand. The ERT is located at the point so that the robotic thumb mirrors the position of the real thumb in the opposite side of the hand.
Denavit–Hartemberg parameters of extra robotic thumb 
Thumb motion capture
The TMC is a device attached to the posterior side of the right hand and the right wrist, and it is used to capture the movements of the right thumb. Figure 2 depicts the overview of the device. This device is designed to follow the movements of the thumb, and record the position of the tip of the thumb with respect to its base. It is designed in such a way to not interfere with the movements of the thumb.
Denavit–Hartemberg parameters of thumb motion Capture 
Electrical stimulation device
Surface electrical stimulation has been shown to be a reliable method to feedback tactile information to users . Electrodes can be very light and easily attached to the location where the stimulation is desired without interfering with the activity of the user. In this study, we attached the electrodes to the tip of the right thumb. Electrical stimulation activates several skin mechanoreceptors depending on the frequency and intensity of the stimulation. With the right parameters, a user can easily and comfortably discriminate between presence and absence, or strength of the stimulation. The developed electrical stimulation system consists of an electric stimulator, a current amplifier, an I/O board, a signal multiplexor, a switching circuit, and electrodes as shown in Fig. 3.
We conducted experiments of a finger reaching task with and without visual feedback from the movement of the ERT in order to evaluate the level of embodiment of ERT.
Experimental results of reaching task
The experiment was performed five times with rest times of 1 day between sessions, then four times with rest time of 1 week, and finally one last time, after 1 month had passed since last session, in order to see if progress is affected, or performance ability forgotten over time.
Experimental results of proprioceptive drift
The subjects were asked to point, as closely as possible, the position of the right thumb tip with the left index finger while having the eyes closed. The subjects were asked to perform this activity before and after each session to see if the use of the sixth finger device had any effect in the self-perception of the subject.
The experimental results of reaching task indicate that the number of channels and type of feedback is an important factor on the level of embodiment that a supernumerary limb device can achieve in a set amount of time. From previous results  we know that the presence of somatosensory feedback is an important factor for the performance of this kind of devices. We assume that the level of embodiment is directly related to the performance in reaching tasks, via the accuracy of the efferent copy of the body schema of the system user-device. If the movements predicted by the feed forward model of the user match the real actions of the sixth finger, then the user will be able to reach more targets in a given time. In our experiments, the group without visual feedback from the sixth finger had an slightly better performance, pointing that they had to rely less on the visual information, and more in the tactile information, which would have helped in creating a better efferent copy of the sixth finger system.
The experimental results of the proprioceptive drift indicate that the use of direct control to drive a supernumerary limb affects not only the creation of a new body representation of the supernumerary limb, but also modifies the existing body representation of the controlling limb. In this case, the self-image of the position of the fingertip is modified during the course of the experiment, pointing to the plasticity of the brain to modify the existing body schema.
There is a light correlation between performance and proprioceptive drift, which could indicate that the process of creating the new body representation of the sixth finger to the movement of the right thumb is related to the accuracy of the right thumb body representation. This could be caused by the brain using the efferent information of the right thumb in creating the efferent copy of the sixth finger, and not in its movement, degrading the efferent copy of the real thumb.
The decrease in accuracy in the body representation of the right thumb is lower, than the increase of performance of the sixth finger. Meaning that the brain is able to adapt to a more complex body schema, introducing new limbs without seriously sacrificing existing parts of the body representation. The limit of the complexity of the body schema achievable is still an open question to be researched.
It has been studied that patients recovering from cervical injuries  or stroke  can regain part of the body schema lost due to injury or disease . It has also been studied the use of extra robotic fingers to help patients recovering from stroke . Our results indicate that it might be possible to help these kinds of patients to achieve a new body schema by sacrificing other part of the body schema, specifically, part that is no longer in use.
We presented a sixth finger system composed of an extra robotic thumb attached to the left hand, that simulates a second thumb in that hand and has a force sensor in its tip; a thumb motion capture, that attaches to the right hand, and captures the movements and position of the tip of the thumb in that hand, to mirror its movements into the extra robotic thumb. And a electrostimulation device that provides pseudo tactile feedback from the force sensor on the tip of the robotic thumb to the tip of the right thumb.
We showed the performance of this sixth finger device with somatosensory feedback measured via a pointing task by comparing two groups of subjects; the subjects of the first group were allowed to see the sixth finger device during the test, and the subjects of second group were not allowed to see the sixth finger device during the test. The movement of the device was controlled by the opposite hand thumb and the contact information was conveyed via electric stimulation from the tip of the extra robotic thumb to the tip of the finger controlling the movement. The experimental results show the group without visual feedback had a better performance than the group with visual feedback; this indicates that subjects of this group achieved better embodiment of the system in the user’s body schema.
In addition, we measured the proprioceptive drift of the controlling thumb by asking the user to point the location of the tip of the right thumb, before and after the experiment. The results show that the proprioceptive drift is always positive meaning that the accuracy is always lower after the experiment. This results also indicate that the sixth finger is not just embodied, but also controlling the device in a direct manner modifies the controlling thumb schema; there is a light correlation between performance and proprioceptive drift. It is also suggested that the ability of the brain to adapt to different scenarios and geometries of the body even those with supernumerary limbs that were not present before, and the body schema of existing limbs can be modified via somatosensory feedback. The body schema will be modified in a way that resembles transfer of embodiment from the controlling limb to the new supernumerary one by using the efferent information from one movement to control a new limb, and providing afferent information from the actions of the new limb.
Proper embodiment of artificial limbs means that users can control them without visual feedback, and embodiment of extra limbs increase its usefulness and usability as extensions of the human body. This technique can be used to extend the body schema of the user with the addition of supernumerary limbs or help recovery patients to regain lost body schema.
As a future work the effect and comparison of different kind of interfaces on the embodiment of these kind of systems should be tested. Furthermore, usability cases of these kind of devices in daily life, work or medical treatments could be studied. Also the level of embodiment achievable by using extra robotic limbs that do not have a geometry similar to human body remains to be researched.
YH and MS initiated the research, design and perform the experiments. NSM and MS design and build the devices. NSM performed the data analysis, interpretation of the results and wrote the manuscript with the help and review from TA. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Availability of data and materials
The dataset supporting the conclusions of this article is included within the article and its additional files.
This work was supported by JSPS KAKENHI Grant Number JP17H05906.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Murray CD (2004) An interpretative phenomenological analysis of the embodiment of artificial limbs. Disabil Rehabil 26(16):963–973View ArticleGoogle Scholar
- Murray CD (2008) “Embodiment and Prosthetics” in psychoprosthetics. Springer London, London, pp 119–129View ArticleGoogle Scholar
- Parietti F, Chan K, Hunter B, Asada H (2015) Design and control of supernumerary robotic limbs for balance augmentation. In: IEEE international Conference on robotics and automation. ICRA, Seattle, USA; May 2015Google Scholar
- Schaefer M, Heinze HJ, Rotte M (2009) My third arm: shifts in topography of the somatosensory homunculus predict feeling of an artificial supernumerary arm. Hum Brain Mapp 30:1413–1420View ArticleGoogle Scholar
- Llorens-Bonilla B, Parietti F, Asada HH (2012) Demonstration-based control of supernumerary robotic limbs. In: IEEE/RSJ international conference on intelligent robots and systems, 2012, IEEE, Vilamoura, pp 3936–3942Google Scholar
- Wu F, Asada H (2014) Bio-artificial synergies for grasp posture control of supernumerary robotic fingers. In: Robotics: science and systems, MIT Press, Berkeley, pp 12–16Google Scholar
- Wu FY, Asada HH (2016) Implicit and intuitive grasp posture control for wearable robotic fingers: a data-driven method using partial least squares. IEEE Trans Robot 32:176–186View ArticleGoogle Scholar
- Sobajima M, Sato Y, Xufeng W, Hasegawa Y (2015) Improvement of operability of extra robotic thumb using tactile feedback by electrical stimulation. MHSGoogle Scholar
- Holmes NP, Spence C (2004) The body schema and the multisensory representation(s) of peripersonal space. Cognit process 5(2):94–105. https://doi.org/10.1007/s10339-004-0013-3View ArticleGoogle Scholar
- Di Pino G, Maravita A, Zollo L, Guglielmelli E, Di Lazzaro V (2014) Augmentation-related brain plasticity. Front Syst Neurosci 8:109. https://doi.org/10.3389/fnsys.2014.00109.eCollectionGoogle Scholar
- Mulvey MR, Fawkner HJ, Radford HE, Johnson MI (2012) Perceptual embodiment of prosthetic limbs by transcutaneous electrical nerve stimulation. Neuromodulation 15(1):42–46 (discussion 47)View ArticleGoogle Scholar
- Bicchi A, Salisbury JK, Dario P (1989) Augmentation of grasp robustness using intrinsic tactile sensing. In: Proceedings, 1989 international conference on robotics and automation, Scottsdale, AZ, pp 302–307Google Scholar
- Prattichizzo D, Malvezzi M, Hussain I, Salvietti G (2014) The sixth-finger: a modular extra-finger to enhance human hand capabilities. In: The 23rd IEEE international symposium on robot and human interactive communication, Edinburgh, pp 993–998Google Scholar
- Hussain I, Salvietti G, Meli L, Pacchierotti S, Prattichizzo D (2015) Using the robotic sixth finger and vibrotactile feedback for grasp compensation in chronic stroke patients. In: Proceedings IEEE/RAS-EMBS international conference rehabilitation robotics (ICORR)Google Scholar
- Salvietti G, Hussain I, Cioncoloni D, Taddei S, Rossi S, Prattichizzo D (2017) Compensating hand function in chronic stroke patients through the robotic sixth finger. IEEE Trans Neural Syst Rehabil Eng 25(2):142–150View ArticleGoogle Scholar
- Morasso P et al (2017) Revisiting the body-schema concept in the context of whole-body postural-focal dynamics. Front Hum Neurosci 9:83Google Scholar
- Shokur S, et al. (2013) Expanding the primate body schema in sensorimotor cortex by virtual touches of an avatar. In: Proceedings of the national academy of sciences of the United States of America 110(37):15121–15126. PMC. Web. 7 Sept. 2017Google Scholar
- Ramsey R, Cross ES, de Hamilton Antonia AF (2013) Supramodal and modality-sensitive representations of perceived action categories in the human brain. Exp Brain Res 230:345–357View ArticleGoogle Scholar
- Chatterjee A, Aggarwal V, Ramos A, Acharya S, Thakor NV (2007) A brain-computer interface with vibrotactile biofeedback for haptic information. J NeuroEng Rehabil 4(1):40View ArticleGoogle Scholar
- Bay JS (1989) Tactile shape sensing via single- and multifingered hands. In: Proceedings, 1989 international conference on robotics and automation, vol. 1, Scottsdale, AZ, pp 290–295Google Scholar
- Hasegawa Y, Ozawa K (2014) Pseudo-somatosensory feedback about joints angle using electrode array, IEEE/SICE international Symposium on system integration (SII), pp 644–649Google Scholar
- Paul Richard (1981) Robot manipulators: mathematics, programming, and control: the computer control of robot manipulators. MIT Press, CambridgeGoogle Scholar
- Purves D, Augustine GJ, Fitzpatrick D, et al., editors (2001) Neuroscience, 2nd edition. Sinauer Associates, Sunderland. Mechanoreceptors specialized to receive tactile information. https://www.ncbi.nlm.nih.gov/books/NBK10895/
- Perreault EJ, Crago PE, Kirsch RF (2001) Postural arm control following cervical spinal cord injury. IEEE Trans Neural Syst Rehabil Eng 9(4):369–377View ArticleGoogle Scholar
- Volpe BT, Krebs HI, Hogan N (2001) Is robot-aided sensorimotor training in stroke rehabilitation a realistic option? Curr Opin Neurol 14(6):745–752View ArticleGoogle Scholar
- Webb J, Xiao ZG, Aschenbrenner KP (2012) Towards a portable assistive arm exoskeleton for stroke patient rehabilitation controlled through a brain computer interface. In: 4th IEEE RAS and EMBS international conference on biomedical robotics and biomechatronics, Rome, Italy, 2427, pp 1299–1304Google Scholar
- Chiri A, Vitiello N, Giovacchini F, Roccella S, Vecchi F, Carrozza MC (2012) Mechatronic design and characterization of the index finger module of a hand exoskeleton for post-stroke rehabilitation. IEEE/ASME Trans Mech 17(5):884–894View ArticleGoogle Scholar