I. Analysis of the Artificial Intelligence Industry
The development of the artificial intelligence (AI) industry will usher in a new round of competition among cities and regions. According to a tracking study of regional AI technology industry competitiveness evaluation index conducted by the China New Generation Artificial Intelligence Development Strategy Research Institute from 2018 to 2021, the Yangtze River Delta region surpassed the Beijing-Tianjin-Hebei region in total score for the first time in 2021, ranking first. The accelerated integration of AI with the real economy and the southward shift of AI technology innovation resources from the north are important factors changing the regional competitiveness landscape. Therefore, regions should accelerate the completion of the AI and its industry-specific industrial chains, actively build demonstrative smart application scenarios, proactively develop AI-related standards and management systems, promote resource sharing in public R&D, and strengthen scientific research and talent cultivation to seize the significant historical opportunity for the development of the AI industry.
The AI industry is transitioning from its development phase to maturity, with the computer vision market forming a major segment. Except for AI chips, other sub-sectors have moved beyond high-speed growth and entered a phase of steady growth. The AI market is projected to reach 199.8 billion yuan in 2021 and exceed 600 billion yuan by 2026. Computer vision remains the largest contributor to the market, but as downstream stakeholders increasingly embrace digitalization, their demands for data—a key production factor for AI models—are rising. This has led to a surge in demand for data-related products incorporating machine learning technologies, driving the machine learning market to some extent. Furthermore, AI chips, as crucial hardware for the AI industry, are projected to grow at a CAGR of over 40% from 2021 to 2026, significantly contributing to the overall industry's core growth and overall scale growth.
II. Artificial intelligence (AI) cannot function without storage.
Digital transformation has become an essential means for enterprises to upgrade their businesses. In fact, in the decade since the concept of "digital transformation" was proposed, core technologies such as 5G, big data, cloud computing, artificial intelligence, and the Internet of Things have blossomed in various industries. Newer technologies such as edge computing, machine learning, and digital twins are emerging one after another, reshaping business models, disrupting life experiences, and accelerating the intelligence of everything. Tracing back to the source, all changes originate from data and are driven by data.
As digital transformation enters its 2.0 era, enterprises are continuously increasing their investments, hoping that individual technologies can be aggregated into a system and exert an integrated effect to further increase profits, stimulate innovation, improve employee productivity, enhance operational efficiency, and improve customer experience.
While AI is constantly driving the development of storage, further unlocking its potential still requires addressing the challenges storage faces in AI-driven scenarios:
The training tasks require hundreds of millions to billions of files, so the storage needs to be able to handle billions or even tens of billions of files. Furthermore, many training models rely on image, audio, and video clips, which are typically between a few KB and a few MB in size.
In most scenarios, the training task only reads files and rarely generates intermediate data. Even if a small amount of intermediate data is generated, it is usually written locally rather than written back to the storage cluster.
Directory hotspots occur because the data organization methods of business departments are uncontrollable during training. Users may store a large number of files in the same directory, which can easily lead to multiple computing nodes reading this batch of data at the same time during training. This makes the metadata node containing this directory a hotspot.
"To do a good job, one must first have the right tools." Similarly, to unleash the full power of AI technology, addressing storage challenges is a crucial part of building a robust IT infrastructure. Without high-performance storage, the entire system will experience performance delays.
