New Storage Devices and Systems
Editor's Note
Coming soon
Guest Editor
Yu Hua, Professor
Huazhong University of Science and Technology
His research interests include new storage devices, cloud storage systems, non-volatile memory, etc.
Article List
2022, 11(3):3-22.
DOI: 10.12146/j.issn.2095-3135.20211001002
Abstract:
Big data applications have an increasing demand for memory capacity, but traditional memory using DRAM as a memory medium has become more and more serious in big data applications. Computer designers began to consider using Non-Volatile Memory (NVM) to replace traditional DRAM memory. As a non-volatile storage medium, NVM does not need to be dynamically refreshed, so it will not cause a large amount of energy consumption; at the same time, the read performance of NVM is similar to that of DRAM, and the capacity of a single NVM storage unit has strong scalability. However, integrating NVM as a memory into an existing computer system needs to solve its security problem. Traditional DRAM, as a memory medium, loses data automatically after power failure, so the data will not stay in the storage medium for a long time, while NVM is a non-volatile storage medium, and the data can be retained in the NVM for a relatively long time. If attackers gain access to the NVM and then scan the contents, they can obtain the data in the memory. This security issue is defined as a "recovery vulnerability" of the data. Therefore, in a data center environment based on NVM modules, how to make full and effective use of NVM and ensure its safety has become an urgent problem to be solved. Starting from the security aspect of NVM, this article summarizes the research hotspots and progress of NVM security in recent years. First, it summarizes the main security issues faced by NVM, such as data theft, integrity damage, data consistency and crash recovery, and system performance degradation caused by the introduction of encryption and decryption and integrity protection technologies. Then, in view of the above problems, the combined counter mode encryption technology, integrity protection technology Bonsai Merkel Tree, data consistency and crash recovery technology and related optimization schemes are introduced in detail. Finally, the full text is summarized, and the issues that need further attention in the future of NVM are prospected.
Big data applications have an increasing demand for memory capacity, but traditional memory using DRAM as a memory medium has become more and more serious in big data applications. Computer designers began to consider using Non-Volatile Memory (NVM) to replace traditional DRAM memory. As a non-volatile storage medium, NVM does not need to be dynamically refreshed, so it will not cause a large amount of energy consumption; at the same time, the read performance of NVM is similar to that of DRAM, and the capacity of a single NVM storage unit has strong scalability. However, integrating NVM as a memory into an existing computer system needs to solve its security problem. Traditional DRAM, as a memory medium, loses data automatically after power failure, so the data will not stay in the storage medium for a long time, while NVM is a non-volatile storage medium, and the data can be retained in the NVM for a relatively long time. If attackers gain access to the NVM and then scan the contents, they can obtain the data in the memory. This security issue is defined as a "recovery vulnerability" of the data. Therefore, in a data center environment based on NVM modules, how to make full and effective use of NVM and ensure its safety has become an urgent problem to be solved. Starting from the security aspect of NVM, this article summarizes the research hotspots and progress of NVM security in recent years. First, it summarizes the main security issues faced by NVM, such as data theft, integrity damage, data consistency and crash recovery, and system performance degradation caused by the introduction of encryption and decryption and integrity protection technologies. Then, in view of the above problems, the combined counter mode encryption technology, integrity protection technology Bonsai Merkel Tree, data consistency and crash recovery technology and related optimization schemes are introduced in detail. Finally, the full text is summarized, and the issues that need further attention in the future of NVM are prospected.
2022, 11(3):23-41.
DOI: 10.12146/j.issn.2095-3135.20211019001
Abstract:
The isolation of storage and compute units in the Von Neumann architecture leads to the “storage wall” problem, which makes the existing system architecture hard to cope with the challenges of data explosion caused by the wide application of big data and artificial intelligence technologies. The continuous growth of data has led to an evolution in the computing paradigm. Researchers try to move the compute unit to the storage system, that is Near-Data Processing (NDP) technology. NDP technology refers to utilizing the computing power of the storage controller to perform I/O intensive computing tasks, which brings advantages such as low latency, high scalability, and low power consumption while reducing data movement, and has broad application prospects. This article first introduces the near-data computing architecture, subsequently outlines the research results of NDP systems for specific applications and some general scenarios, then summarizes the hardware and software platform and industry progress of NDP, finally looks into the future development trend of NDP technology.
The isolation of storage and compute units in the Von Neumann architecture leads to the “storage wall” problem, which makes the existing system architecture hard to cope with the challenges of data explosion caused by the wide application of big data and artificial intelligence technologies. The continuous growth of data has led to an evolution in the computing paradigm. Researchers try to move the compute unit to the storage system, that is Near-Data Processing (NDP) technology. NDP technology refers to utilizing the computing power of the storage controller to perform I/O intensive computing tasks, which brings advantages such as low latency, high scalability, and low power consumption while reducing data movement, and has broad application prospects. This article first introduces the near-data computing architecture, subsequently outlines the research results of NDP systems for specific applications and some general scenarios, then summarizes the hardware and software platform and industry progress of NDP, finally looks into the future development trend of NDP technology.
2022, 11(3):42-55.
DOI: 10.12146/j.issn.2095-3135.20211017001
Abstract:
With the development of big data and artificial intelligence applications, data are growing explosively, and the demand for data storage is increasing day by day. The capacity of traditional memory technology is approaching the limit of its physical storage density. Non-volatile memory is expected to replace traditional dynamic random access memory or disk technology due to its excellent characteristics such as byte addressability, low energy consumption, and fast read and write speed. However, the storage medium itself has some shortcomings, such as limited lifetime, asymmetric read and write speed, uneven wear and various sources of errors. The storage principles of common non-volatile memories are explained, and existing improved technologies are investigated and summarized.
With the development of big data and artificial intelligence applications, data are growing explosively, and the demand for data storage is increasing day by day. The capacity of traditional memory technology is approaching the limit of its physical storage density. Non-volatile memory is expected to replace traditional dynamic random access memory or disk technology due to its excellent characteristics such as byte addressability, low energy consumption, and fast read and write speed. However, the storage medium itself has some shortcomings, such as limited lifetime, asymmetric read and write speed, uneven wear and various sources of errors. The storage principles of common non-volatile memories are explained, and existing improved technologies are investigated and summarized.
2022, 11(3):56-70.
DOI: 10.12146/j.issn.2095-3135.20210913001
Abstract:
Non-volatile memory has a read/write speed that is comparable to dynamic random access memory and can be used to replace traditional storage devices to improve the performance of storage engines. However, existing storage engines typically use generic block interfaces to access devices, resulting in a long I/O software stack, increasing read/write latency at software layers, thereby limiting the performance benefits of non-volatile memory. To solve this problem, this paper proposes a new storage engine, named NVMStore, which is based on non-volatile memory and the Ceph big-data storage system platform. NVMStore accesses storage devices through memory mapping and optimizes data read/write processes according to byte-addressability and data persistence characteristics of non-volatile memory, thus reducing the data write amplification and software stack overhead. Experimental results on real non-volatile memory devices show that NVMStore can significantly improve the performance of Ceph when dealing with small block data read/ write workloads, compared with traditional storage engines using non-volatile memory.
Non-volatile memory has a read/write speed that is comparable to dynamic random access memory and can be used to replace traditional storage devices to improve the performance of storage engines. However, existing storage engines typically use generic block interfaces to access devices, resulting in a long I/O software stack, increasing read/write latency at software layers, thereby limiting the performance benefits of non-volatile memory. To solve this problem, this paper proposes a new storage engine, named NVMStore, which is based on non-volatile memory and the Ceph big-data storage system platform. NVMStore accesses storage devices through memory mapping and optimizes data read/write processes according to byte-addressability and data persistence characteristics of non-volatile memory, thus reducing the data write amplification and software stack overhead. Experimental results on real non-volatile memory devices show that NVMStore can significantly improve the performance of Ceph when dealing with small block data read/ write workloads, compared with traditional storage engines using non-volatile memory.
2022, 11(3):71-84.
DOI: 10.12146/j.issn.2095-3135.20220225001
Abstract:
Hybrid flash storage has become the mainstream storage device in the field of consumer device. However, the academic study on hybrid flash storage is still insufficient. Based on our research activities, practical experience on hybrid storage devices, and state-of-the-art researches, this paper introduces the architecture of hybrid flash memory, the pain points that need to be solved and the relevant research progress. Firstly, this paper introduces and analyses the hybrid flash memory architecture and the corresponding characteristics. Then the experimental results on real hybrid flash memory are shown and the problems of the hybrid flash memory to be solved are exposed. These problems are full into four categories, write characteristics, read characteristics, read/write interference, and volume characteristics. Finally, the latest research progresses of the corresponding problems are introduced. The advantages and disadvantages of each technique are summarized. Additionally, the future development direction is commented.
Hybrid flash storage has become the mainstream storage device in the field of consumer device. However, the academic study on hybrid flash storage is still insufficient. Based on our research activities, practical experience on hybrid storage devices, and state-of-the-art researches, this paper introduces the architecture of hybrid flash memory, the pain points that need to be solved and the relevant research progress. Firstly, this paper introduces and analyses the hybrid flash memory architecture and the corresponding characteristics. Then the experimental results on real hybrid flash memory are shown and the problems of the hybrid flash memory to be solved are exposed. These problems are full into four categories, write characteristics, read characteristics, read/write interference, and volume characteristics. Finally, the latest research progresses of the corresponding problems are introduced. The advantages and disadvantages of each technique are summarized. Additionally, the future development direction is commented.
2022, 11(3):85-97.
DOI: 10.12146/j.issn.2095-3135.20211026001
Abstract:
With the explosive growth of Internet data, many distributed storage systems have integrated erasure-coding mechanisms to ensure data reliability, while further reducing storage overhead. However, erasure-coding has changed the data placement scheme, thus affecting the data access of other services of the cluster. This paper proposes a new data placement scheme and a task scheduling strategy based on heterogeneous Hadoop cluster that can be better adapted to the “one-to-many” data access scenarios of a typical offline batch job——MapReduce applications. By analyzing the hardware parameters and historical load of each node in a heterogeneous cluster, the data blocks of the same erasure coded stripe are distributed as many as possible on nodes with similar performance. This way ensures that the data access pressure to each node of the cluster during the execution of the MapReduce job achieves relatively balanced state. In addition, when the system schedules tasks, the task concurrency of nodes is determined according to the current load and computing power of each node and so to avoid straggler task caused by heavy load in some nodes and optimize the progress of the MapReduce job. The experimental results show that compared with the default random data placement and task allocation strategy in Hadoop, the data layout strategy Heterogeneous-aware Data Placement Algorithm (HDPA) and the task allocation strategy Dynamic Task Allocation Algorithm (DTAA) proposed in this paper can effectively reduce the long tail effect of tasks in different types of MapReduce applications, thus reducing the running time by 10.5%~42%.
With the explosive growth of Internet data, many distributed storage systems have integrated erasure-coding mechanisms to ensure data reliability, while further reducing storage overhead. However, erasure-coding has changed the data placement scheme, thus affecting the data access of other services of the cluster. This paper proposes a new data placement scheme and a task scheduling strategy based on heterogeneous Hadoop cluster that can be better adapted to the “one-to-many” data access scenarios of a typical offline batch job——MapReduce applications. By analyzing the hardware parameters and historical load of each node in a heterogeneous cluster, the data blocks of the same erasure coded stripe are distributed as many as possible on nodes with similar performance. This way ensures that the data access pressure to each node of the cluster during the execution of the MapReduce job achieves relatively balanced state. In addition, when the system schedules tasks, the task concurrency of nodes is determined according to the current load and computing power of each node and so to avoid straggler task caused by heavy load in some nodes and optimize the progress of the MapReduce job. The experimental results show that compared with the default random data placement and task allocation strategy in Hadoop, the data layout strategy Heterogeneous-aware Data Placement Algorithm (HDPA) and the task allocation strategy Dynamic Task Allocation Algorithm (DTAA) proposed in this paper can effectively reduce the long tail effect of tasks in different types of MapReduce applications, thus reducing the running time by 10.5%~42%.
