Distributed metadata consistency is one of the critical issues of metadata clusters in distributed file systems. Existing methods to maintain metadata consistency generally need several log forced write operations. Si...Distributed metadata consistency is one of the critical issues of metadata clusters in distributed file systems. Existing methods to maintain metadata consistency generally need several log forced write operations. Since synchronous disk IO is very inefficient, the average response time of metadata operations is greatly increased. In this paper, an asynchronous atomic commit protocol (ACP) named Dual-Log (DL) is presented. It does not need any log forced write operations. Optimizing for distributed metadata operations involving only two metadata servers, DL mutually records the redo log in counterpart metadata servers by transferring through the low latency network. A crashed metadata server can redo the metadata operation with the redundant redo log. Since the latency of the network is much lower than the latency of disk IO, DL can improve the performance of distributed metadata service significantly. The prototype of DL is implemented based on local journal. The performance is tested by comparing with two widely used protocols, EP and S2PC-MP, and the results show that the average response time of distributed metadata operations is reduced by about 40%-60%, and the recovery time is only I second under 10 thousands uncompleted distributed metadata operations.展开更多
在线事务处理(online transaction processing,OLTP)应用面临并发量和数据量持续增长的问题,并且高并发读写操作使得后台数据库成为瓶颈。内存数据网格(in-memory data grid,IMDG)是基于内存的新型分布式数据访问平台,是解决系统数据库...在线事务处理(online transaction processing,OLTP)应用面临并发量和数据量持续增长的问题,并且高并发读写操作使得后台数据库成为瓶颈。内存数据网格(in-memory data grid,IMDG)是基于内存的新型分布式数据访问平台,是解决系统数据库写操作瓶颈的有效技术途径之一。然而内存数据网格中数据访问操作涉及的数据分布是不可预知的,需要提供分布式事务保障。针对内存数据网格的系统特点,提出了一种分布式事务保障机制,设计实现了事务处理模型、请求处理和数据定位方法以及事务保障协议,并规范化地定义了客户端与服务器端以及服务器端之间的操作接口。在事务处理基准测试TPC-W上的实验结果表明,新机制可以提高在线事务应用的处理速度,并具备良好的扩展性。展开更多
基金supported by the National Basic Research 973 Program of China under Grant No.2011CB302304the NationalHigh Technology Research and Development 863 Program of China under Grant Nos.2011AA01A102 and 2013AA013205+1 种基金the StrategicPriority Research Program of the Chinese Academy of Sciences under Grant No.XDA06010401the Chinese Academy of SciencesKey Deployment Project under Grant No.KGZD-EW-103-5(7)
摘要Distributed metadata consistency is one of the critical issues of metadata clusters in distributed file systems. Existing methods to maintain metadata consistency generally need several log forced write operations. Since synchronous disk IO is very inefficient, the average response time of metadata operations is greatly increased. In this paper, an asynchronous atomic commit protocol (ACP) named Dual-Log (DL) is presented. It does not need any log forced write operations. Optimizing for distributed metadata operations involving only two metadata servers, DL mutually records the redo log in counterpart metadata servers by transferring through the low latency network. A crashed metadata server can redo the metadata operation with the redundant redo log. Since the latency of the network is much lower than the latency of disk IO, DL can improve the performance of distributed metadata service significantly. The prototype of DL is implemented based on local journal. The performance is tested by comparing with two widely used protocols, EP and S2PC-MP, and the results show that the average response time of distributed metadata operations is reduced by about 40%-60%, and the recovery time is only I second under 10 thousands uncompleted distributed metadata operations.
摘要在线事务处理(online transaction processing,OLTP)应用面临并发量和数据量持续增长的问题,并且高并发读写操作使得后台数据库成为瓶颈。内存数据网格(in-memory data grid,IMDG)是基于内存的新型分布式数据访问平台,是解决系统数据库写操作瓶颈的有效技术途径之一。然而内存数据网格中数据访问操作涉及的数据分布是不可预知的,需要提供分布式事务保障。针对内存数据网格的系统特点,提出了一种分布式事务保障机制,设计实现了事务处理模型、请求处理和数据定位方法以及事务保障协议,并规范化地定义了客户端与服务器端以及服务器端之间的操作接口。在事务处理基准测试TPC-W上的实验结果表明,新机制可以提高在线事务应用的处理速度,并具备良好的扩展性。