Abstract: This paper introduces a non-humanoid percussion robot. The robot works similarly to playing the piano, using a robotic arm to strike the keys. The entire system is controlled by an STM32 microcontroller, with the mechanical striking arm using servo motors as its movable joints and controlled by the chip. It features human-machine interaction, allowing users to control the robot and manage the system's music program files through the user interface. Users can not only save and modify these files but also customize their content. Furthermore, the interface supports user-defined program files. During performance, the contents of these files are processed by the microcontroller, translated into instructions, and then used to control the motors to strike the keys and produce music via a serial port.
1. Introduction
As society becomes increasingly intelligent, more and more robots are appearing in our lives, permeating everything from industry and agriculture to daily living. Robots have transformed from futuristic science fiction into indispensable parts of our lives. Intelligentization has been a hot topic in recent years, with many products developing in this direction to provide users with a comfortable and convenient experience. While there are continuous breakthroughs in intelligent robots, few products in the field of music offer human-computer interaction, allowing users to truly experience the robot's ease of use and engaging nature. Typical music players simply play audio, but the intelligent music robot we're developing will perform live, producing a more realistic sound than what you'd hear from an amplifier. It features a mechanical percussion arm controlled by a servo motor, rotating around a pivot point and striking the music up and down. Below it is a soundboard with different keys, allowing it to play like a piano. Another highlight is its human-computer interaction function, allowing users to program and compose music through the interface.
2. Overall Plan
2.1 System Overall Structure
The music robot mainly consists of two parts: its mechanical structure and its control system. The mechanical structure primarily includes a mechanical striking arm, a soundboard, and an external fixed frame. The robotic arm employs dual-degree-of-freedom control, using two servo motors to control the movement of two joints respectively. The soundboard has keys fixed to it, and the striking arm plays music by striking these keys. The control system's function is to control the mechanical structure according to instructions. It uses a touchscreen LCD as the human-machine interface, and the control unit uses an STM32 microcontroller as its core, responsible for storing files, reading internal music files, translating the music program the user needs to execute, and driving the motors to play the music.
2.2 Working Principle
The robot's ability to perform perfectly largely relies on its control system, which is also the focus of this project. Using the user interface to write and read music files is similar to the "write and save file" and "open and read file" functions on a computer. The music written by the user through the interface is saved by the system. When needed for performance, a specific file is selected. When executing the file, the system translates the file content into control instructions, essentially inputting programming instructions into the robot. Following these instructions, the robot controls the striking arm to strike the soundboard. This is the overall concept behind the music robot's operation. The entire control system can be broken down into different functional units. To enable the robot's operation, there are mainly a main control unit, a power supply unit, a motor drive unit, a human-machine interaction unit, and a file management unit. The main control unit controls the entire mechanism; the power supply unit provides the appropriate voltage for the entire system; the human-machine interaction unit allows the user to operate the machine, browse, edit, and execute files, and also supports functions such as pausing and switching files during performance; the file management unit is used to save, retrieve, and manage files; and the motor drive unit receives commands from the control chip to control the motors to cooperate with the performance. The overall structural block diagram is shown in Figure 1.
Figure 1. Control principle diagram of the music robot
3. Robot structural design
The actuator is mechanical, using a striking arm to strike the soundboard to produce sound. The striking arm uses two servo motors as movable joints; one allows it to rotate around a point in the horizontal plane, and the other allows it to strike up and down in the vertical direction, as shown in Figure 2. Because the striking arm's range of motion is fan-shaped, the keys corresponding to each note on the soundboard are arranged in a ring around the striking arm.
Figure 2. Structure of the robotic arm
The striking arm is made of wood, which is lightweight, inexpensive, and gentle on the soundboard with each strike. The soundboard's note keys are made of aluminum and can be selected with different pitch levels, such as seven or eight notes. The soundboard's structure is similar to a seven-tone piano, with a wooden frame and aluminum keys, using different key lengths to produce different notes. A servo motor is used for the drive motor; servos are the optimal choice in terms of both range of motion and control precision.
4. Implementation of robot control methods
Once the robot is turned on and initialized, an operation interface will appear on the screen, supporting the writing of simple text. Users can create new files, write and save them, or select files to execute. During execution, the text will be translated into corresponding musical notes, and the rhythm and pauses can be determined by the text content.
4.1 Hardware Selection
1) Main control unit
The control chip is the core of the entire control system; almost all tasks are scheduled by the chip, including displaying, editing, and storing files, controlling the human-machine interface, executing programs corresponding to files, and controlling motors. Considering all factors, we chose the STM32F407 chip from STMicroelectronics. The STM32F407 is a high-performance, low-cost, and low-power chip, using a Cortex-M4 core with a high clock speed of 168MHz (achieving a processing power of 210 DMIPS). Its main advantages are as follows:
a) Low price.
b) A more advanced core. The STM32F4 uses a Cortex-M4 core with FPU and DSP instruction sets.
c) Enhanced peripheral functionality. For the same peripheral section, the STM32F4 features faster analog-to-digital conversion speed, lower ADC/DAC operating voltage, 32-bit timers, a real-time clock (RTC) with calendar function, significantly enhanced I/O multiplexing capabilities, 4KB of battery-backed SRAM, and faster USART and SPI communication speeds.
d) Online debugging is possible. No emulator is required; code can be downloaded using only a serial port.
2) Human-computer interaction unit
The user interface uses a TFT-LCD resistive touchscreen. Compared to the capacitive touchscreen commonly used in mobile phones, the resistive touchscreen has poor light transmission and does not support multi-touch, but it has higher precision, stronger anti-interference ability and stability, and lower requirements for the working environment. It can work normally in dusty or other unfriendly environments, and it is cheaper. The ADS7843 is used as the control chip.
3) File Management Unit
There are many external storage devices on the market, among which the SD card is the most suitable for microcontroller systems. The SD card (Secure Digital Memory Card) is widely used in portable devices such as digital cameras, personal digital assistants (PDAs), and multimedia players. It not only offers large capacities (32Gb and above) but also supports the SPI interface for easy portability and is available in several sizes to meet the requirements of different applications. It boasts high memory capacity, fast data transfer rates, great portability, and excellent security.
4) Motor control unit
For the servo motor selection, digital servos were chosen. Compared to traditional analog servos, digital servos can process the input signal according to set parameters before sending the power pulse to the servo motor; moreover, digital servos have a higher pulse frequency, which means they are faster and more accurate. They also have advantages such as anti-vibration.
5) Power supply unit
The entire operating system requires 5V and has an internal power conversion switch to power the control chip and servo motors. It can be directly connected to AC power. There is also a rechargeable battery as a backup power source; when using AC power, the battery is charging, and when it runs out of power, it can be opened to power the robot.
4.2 Control System Software Design
1) Main operating procedures
After pressing the power button, the system program begins initialization. If it fails, a restart will be prompted. Once initialization is successful, the entire system can be operated via the touchscreen. Files on the SD card can be viewed on the screen, allowing for editing, saving, or creating new files. Other operations can be performed on the screen while playing, such as pausing or stopping the currently running file. The main process is shown in Figure 3.
Figure 3 Control System Flowchart
2) File Management System
The system responsible for reading and managing files on external storage devices uses FatFs, a completely free and open-source FAT file system module designed specifically for small embedded systems. Written entirely in standard C, it boasts excellent hardware platform independence and can be ported to microcontrollers such as the 8051 and ARM series with only minor modifications. It supports multiple storage media and has independent buffers, allowing for reading and writing of multiple files. Users don't need to understand FatFs' internal structure or the complex FAT protocol; as long as they know how to use the application interface functions provided by FatFs, such as f_open (open/create a file), f_close (close a file), f_read (read a file), f_write (write a file), and f_rename (rename/move a file or folder), they can use it even without any C language knowledge—it's as simple as file operations on a computer.
3) Text content translation
The MIDI (Musical Instrument Digital Interface) was proposed in the early 1980s to solve the communication problems between electronic musical instruments. MIDI is the most widely used music standard format in the music production industry and can be called "computer-understandable sheet music." It uses digital control signals for notes to record music. A complete MIDI music piece is only a few tens of kilobytes in size but can contain dozens of music tracks. Almost all modern music is produced and synthesized using MIDI and sound libraries. MIDI does not transmit sound signals, but rather instructions such as notes and control parameters; it instructs the device what to do and how to do it, such as which note to play and at what volume. These are uniformly represented as MIDI messages.
Drawing inspiration from this, we use txt format text files as the music storage files. The text can be converted from sheet music, and even if users want to write their own, the rules are simple: use various numbers and symbols to represent the information in the sheet music. A simple example of the sheet music and its corresponding text file content is shown in Figure 4.
Figure 4 shows the song sheet and its corresponding TXT text.
The system assigns coordinates to each note key on the soundboard, ranging from 0 degrees to N degrees, with each note corresponding to a coordinate. In the file, numbers correspond to different notes. When the file is executed, the system converts each number into an angle to control the servo motor. When a note needs to be struck, the control chip sends a command to rotate the servo motor to the corresponding angle. The tempo is controlled by the servo motor's rotation speed. The soundboard can be replaced; selecting a soundboard with more note keys allows for the playing of more complex pieces.
5. Summary
This paper introduces the design scheme and control method of a percussion-type music playing robot. It can perform desired music live, offering a highly novel experience. The movable percussion arm uses servo motors to precisely control its rotation angle and striking amplitude, ensuring error-free performance. Both user operation of the robot and the use of the file management system are simple and easy to understand, as are the rules for writing music program files. This project combines knowledge from mechanical design, motor control, and even music, fully meeting the requirements for simple music performance while possessing a certain novelty. Testing has verified that the robot performs well when playing simple pieces, fully meeting the design requirements and combining entertainment and educational functions.
