5358

Audio-motor interactions during musical playing with an external timing reference
Shu-Chi Pai1, Ying-Hua Chu1, Hui-Chuan Chang2, and Jo-Fu Lin1

1Institute of Biomedical Engineering, National Taiwan University, Taipei, Taiwan, Taipei, Taiwan, 2College of Humanities and Social Sciences, Taipei Medical University, Taipei, Taiwan

Synopsis

Previous studies have demonstrated brain activation patterns during musical playing. However, while musical playing with multiple sources of auditory inputs is essential for musical practices and group performance, it is less understood. By using an MR-compatible piano keyboard, the present study compared musical playing with or without acoustic feedback or external timing reference. Functional MRI contrasts showed BOLD signal increase in bilateral Superior Temporal Gyrus and BA 42 during musical playing with an external timing reference, which indicates a neuronal processing pattern relating to the coordination of multiple auditory inputs.

Purpose

Playing music requires motor control with high temporal precision1, which is based on fast feedforward and feedback interactions between the motor and the auditory areas2,3. Specifically, feedforward processes are related to motor planning and execution to generate expected acoustic outcomes4,5, while feedback processes are related to precise motor controlling based on the discrepancy between the expected and the actual auditory outcomes6.

In group performances, an external timing reference, e.g. metronome, is essential for each member of a band to synchronize with the other players. Based on the audio-motor interaction model shown in Figure 1, playing the same melody with different acoustic inputs may involve identical feedforward but different feedback mechanisms. The feedback mechanisms involved under different auditory conditions are shown in Figure 2. When an external timing reference is available, the discrepancies between the external reference and expected acoustic outcome are explicitly fed back to the player to improve motor precision and the discrepancy between the external reference and the actual acoustic outcome is also important for feedback control. The present study aims to demonstrate the brain regions involved in the feedback control with external timing reference.

Method

All data were acquired on a 3T MRI scanner (Skyra, Siemens, Erlangen, Germany) with a 32-channel head coil array. BOLD signals were measured by an EPI (FOV: 224 × 224 mm2; resolution: 3.5 mm isotropic; TR = 2000 ms; TE = 30 ms; flip angle = 90o). Anatomical images were acquired using a T1-weighted MPRAGE pulse sequence. We designed an MR-compatible piano by modifying a plastic melody horn (SUZUKI, MELODION, MX-37C) and integrated a Bluetooth module (AIROHA, Bluetooth HID, AB1112A; Figure 3). To verify the image quality in the presence of the MR-compatible piano, the time-domain signal to noise ratio (tSNR) maps were calculated from EPI time courses. Figure 4 shows the tSNR ratio over a reference tSNR value. We recruited 17 subjects (10 females; 21.5 ± 1.5 years) with basic piano skills. They were asked to play the first 15 segments of the first portion of Sonatina Op. 36, No. 1 by Muzio Clementi using the MR-compatible piano inside MRI. Subjects played the music at a constant tempo of 90 beats per minute (90 BPM). The total playing time was 20 seconds. There were four experiment conditions. The first condition had no audio output from the piano (None), which was taken as the case without any audio-motor interaction. In the second condition, players only heard the sound of a metronome (M), which provided the external timing reference. In the third condition, players only heard the audio output of the piano (P). Players heard both metronome and piano output in the 4th condition (MP). Four seconds of metronome beeps were presented to players to notify the beginning of piano playing, which was followed by a 20-s silent resting period. Piano performances were recorded. Trials with unacceptable accuracy were excluded from analysis.

Results

Figure 5 shows the contrasts of four conditions. Contrasting between M and None conditions revealed significant BOLD signals at bilateral Superior Temporal Gyrus (STG) in the temporal lobe and BA 40 in the left parietal lobe. Contrasting between MP and P conditions demonstrated larger brain activity distribution area at bilateral STG and bilateral BA 42 in the temporal lobe. Both contrasts demonstrated similar BOLD signal patterns at bilateral STG in the temporal lobe, but the MP vs P contrast shows more distributed brain activation volume.

Discussion

In the present study, subjects were required to play the same melody under different auditory input conditions, which involved different feedback processes. As shown in Figure 2, contrasting between M and None conditions reveals brain region involved the feedback control after the comparison between the expected acoustic outcome and the external timing reference. While contrasting between MP and P also showed this feedback control, it specifically elucidated brain regions involved in the feedback control based on the discrepancy between the actual acoustic outcome and the external timing reference. Compared to the M vs. None contrast, the MP vs P contrast shows more distributed brain activations, which may be associated with feedback control based on the discrepancy between the external timing reference and the actual acoustic outcome. Overall, we show that playing music with an external timing reference may induce more distributed activations in brain regions responsible for multiple sources of auditory feedback control.

Acknowledgements

This work was partially supported by Ministry of Science and Technology, Taiwan (103-2628-B-002-002-MY3, 105-2221-E-002-104), and the Academy of Finland (No. 298131).

References

1. Zatorre, R.J., J.L. Chen, and V.B. Penhune, When the brain plays music: auditory-motor interactions in music perception and production. Nat Rev Neurosci, 2007. 8(7): p. 547-58.

2. Baumann, S., et al., A network for audio-motor coordination in skilled pianists and non-musicians. Brain Res, 2007. 1161: p. 65-78.

3. Furuya, S. and J.F. Soechting, Role of auditory feedback in the control of successive keystrokes during piano playing. Exp Brain Res, 2010. 204(2): p. 223-37.

4. Pearce, M.T. and G.A. Wiggins, Auditory expectation: the information dynamics of music perception and cognition. Top Cogn Sci, 2012. 4(4): p. 625-52.

5. Sammler, D., et al., Syntax in a pianist's hand: ERP signatures of "embodied" syntax processing in music. Cortex, 2013. 49(5): p. 1325-39.

6. Jancke, L., The dynamic audio-motor system in pianists. Ann N Y Acad Sci, 2012. 1252: p. 246-52.

7. Meister, I.G., et al., Playing piano in the mind--an fMRI study on music imagery and performance in pianists. Brain Res Cogn Brain Res, 2004. 19(3): p. 219-28.

8. Baumann, S., et al., A network for sensory-motor integration: what happens in the auditory cortex during piano playing without acoustic feedback? Ann N Y Acad Sci, 2005. 1060: p. 186-8.

9. Bangert, M., et al., Shared networks for auditory and motor processing in professional pianists: Evidence from fMRI conjunction. Neuroimage, 2006. 30(3): p. 917-926.

Figures

The figure shows a possible model of audio-motor interaction during musical playing with external timing reference. The sounds of the piano are served as the acoustic outcome and the sounds of a metronome are served as the external timing reference. Feedforward motor planning subserves the generation of expected acoustic outcome. Feedback control is based on the fast comparison of the expected acoustic outcome, the actual acoustic outcome, and the external timing reference. Previous researches7-9 have demonstrated the roles of posterior motor cortex (PMC) and supplementary motor area (SMA) in motor planning and STG in auditory perception during musical playing.

P represents the sounds of a piano serving as the actual acoustic outcome and M represents the sounds of a metronome serving as the external timing reference. The None condition involves no audio stimuli, suggesting no audio-motor interaction. Under P condition, only P was provided and feedback control is based on P and the expected acoustic outcome. Under the M condition, only M was provided, and feedback control is based on M and the expected acoustic outcome. The MP condition involves both M and P and feedback control is based on the expected acoustic outcome, M, and P.

An MR-compatible piano. All metal components in the melody horn were replaced with MR-compatible rubber bands and plastic screws. The Bluetooth module was attached under the melody horn. Every key on the keyboard was linked to the Bluetooth module with nonmagnetic bronze wire. Each keyboard pressed generates a corresponding signal to the Bluetooth host. With online virtual MIDI software FreePiano (http://freepiano.tiwb.com), signals sent by the Bluetooth module were transformed into their matching piano sounds.

Figure 4 shows the tSNR ratio of the center slice at different distance from the phantom. The tSNR with only the phantom is served as a reference. Five sets of tSNR values with the piano next to the phantom at the distance of 0cm, 20cm, 40cm, 60cm and 80cm are further calculated the tSNR ratio over the reference. During the experiments, all subjects played the piano at a distance about 60 to 70 cm from the phantom.

(A) M vs None: Bilateral STG in the temporal lobe and BA 40 in the left parietal lobe. MP vs P: Bilateral STG in the temporal lobe and bilateral BA 42 in the parietal lobe. The paired t-test threshold was taken as a cluster size = 5 and uncorrected p < 0.005.

(B) The volume size of STG in both contrasts with peak coordinates and t-test value.


Proc. Intl. Soc. Mag. Reson. Med. 25 (2017)
5358