Container-based virtualization is increasingly popular in cloud computing due to its efficiency and flexibility.Isolation is a fundamental property of containers and weak isolation could cause significant performance ...Container-based virtualization is increasingly popular in cloud computing due to its efficiency and flexibility.Isolation is a fundamental property of containers and weak isolation could cause significant performance degradation and security vulnerability.However,existing works have almost not discussed the isolation problems of system log which is critical for monitoring and maintenance of containerized applications.In this paper,we present a detailed isolation analysis of system log in current container environment.First,we find several system log isolation problems which can cause significant impacts on system usability,security,and efficiency.For example,system log accidentally exposes information of host and co-resident containers to one container,causing information leakage.Second,we reveal that the root cause of these isolation problems is that containers share the global log configuration,the same log storage,and the global log view.To address these problems,we design and implement a system named private logs(POGs).POGs provides each container with its own log configuration and stores logs individually for each container,avoiding log configuration and storage sharing,respectively.In addition,POGs enables private log view to help distinguish which container the logs belong to.The experimental results show that POGs can effectively enhance system log isolation for containers with negligible performance overhead.展开更多
One particular challenge for large‑scale software systems is anomaly detection.System logs are a straightforward and common source of information for anomaly detection.Existing log‑based anomaly detectors are unusable...One particular challenge for large‑scale software systems is anomaly detection.System logs are a straightforward and common source of information for anomaly detection.Existing log‑based anomaly detectors are unusable in real‑world industrial systems due to high false‑positive rates.In this paper,we incorporate human feedback to adjust the detection model structure to reduce false positives.We apply our approach to two industrial large‑scale systems.Results have shown that our approach performs much better than state‑of‑the-art works with 50%higher accuracy.Besides,human feedback can reduce more than 70%of false positives and greatly improve detection precision.展开更多
As computer data grows exponentially,detecting anomalies within system logs has become increasingly important.Current research on log anomaly detection largely depends on log templates derived from log parsing.Word em...As computer data grows exponentially,detecting anomalies within system logs has become increasingly important.Current research on log anomaly detection largely depends on log templates derived from log parsing.Word embedding is utilized to extract information from these templates.However,this method neglects a portion of the content within the logs and confronts the challenge of data imbalance among various log template types after parsing.Currently,specialized research on data imbalance across log template categories remains scarce.A dual-attention-based log anomaly detection model(LogDA),which leveraged data imbalance,was proposed to address these issues in the work.The LogDA model initially utilized a pre-trained model to extract semantic embedding from log templates.Besides,the similarity between embedding was calculated to discern the relationships among the various templates.Then,a Transformer model with a dual-attention mechanism was constructed to capture positional information and global dependencies.Compared to multiple baseline experiments across three public datasets,the proposed approach could improve precision,recall,and F1 scores.展开更多
Log anomaly detection is an important paradigm for system troubleshooting.Existing log anomaly detection based on Long Short-Term Memory(LSTM)networks is time-consuming to handle long sequences.Transformer model is in...Log anomaly detection is an important paradigm for system troubleshooting.Existing log anomaly detection based on Long Short-Term Memory(LSTM)networks is time-consuming to handle long sequences.Transformer model is introduced to promote efficiency.However,most existing Transformer-based log anomaly detection methods convert unstructured log messages into structured templates by log parsing,which introduces parsing errors.They only extract simple semantic feature,which ignores other features,and are generally supervised,relying on the amount of labeled data.To overcome the limitations of existing methods,this paper proposes a novel unsupervised log anomaly detection method based on multi-feature(UMFLog).UMFLog includes two sub-models to consider two kinds of features:semantic feature and statistical feature,respectively.UMFLog applies the log original content with detailed parameters instead of templates or template IDs to avoid log parsing errors.In the first sub-model,UMFLog uses Bidirectional Encoder Representations from Transformers(BERT)instead of random initialization to extract effective semantic feature,and an unsupervised hypersphere-based Transformer model to learn compact log sequence representations and obtain anomaly candidates.In the second sub-model,UMFLog exploits a statistical feature-based Variational Autoencoder(VAE)about word occurrence times to identify the final anomaly from anomaly candidates.Extensive experiments and evaluations are conducted on three real public log datasets.The results show that UMFLog significantly improves F1-scores compared to the state-of-the-art(SOTA)methods because of the multi-feature.展开更多
In this paper we have developed a data logging and monitoring system, we validated the system by comparing the result from it with the existing one and found that the system performs slightly better than the existing ...In this paper we have developed a data logging and monitoring system, we validated the system by comparing the result from it with the existing one and found that the system performs slightly better than the existing work in the same area. This implies that the data logger and monitoring system is good and can be used to monitor solar energy variables even at the comfort of our homes. We fitted a model to the generated data and found that the meteorological variables considered accounted for 99.88% of the power output in the rainy seasons while 0.12% of the variation was not explained due to other factors. Solar panels inclined at an angle of 5° (Tilt) and facing South Pole perform optimally.展开更多
Microservices have become popular in enterprises because of their excellent scalability and timely update capabilities.However,while fine-grained modularity and service-orientation decrease the complexity of system de...Microservices have become popular in enterprises because of their excellent scalability and timely update capabilities.However,while fine-grained modularity and service-orientation decrease the complexity of system development,the complexity of system operation and maintenance has been greatly increased,on the contrary.Multiple types of system failures occur frequently,and it is hard to detect and diagnose failures in time.Furthermore,microservices are updated frequently.Existing anomaly detection models depend on offline training and cannot adapt to the frequent updates of microservices.This paper proposes an anomaly detection approach for microservice systems with multi-source data streams.This approach realizes online model construction and online anomaly detection,and is capable of self-updating and self-adapting.Experimental results show that this approach can correctly identify 78.85%of faults of different types.展开更多
基金supported by the National Key R&D Program(2022YFB4500704)the National Natural Science Foundation of China(Grant No.62032008).
摘要Container-based virtualization is increasingly popular in cloud computing due to its efficiency and flexibility.Isolation is a fundamental property of containers and weak isolation could cause significant performance degradation and security vulnerability.However,existing works have almost not discussed the isolation problems of system log which is critical for monitoring and maintenance of containerized applications.In this paper,we present a detailed isolation analysis of system log in current container environment.First,we find several system log isolation problems which can cause significant impacts on system usability,security,and efficiency.For example,system log accidentally exposes information of host and co-resident containers to one container,causing information leakage.Second,we reveal that the root cause of these isolation problems is that containers share the global log configuration,the same log storage,and the global log view.To address these problems,we design and implement a system named private logs(POGs).POGs provides each container with its own log configuration and stores logs individually for each container,avoiding log configuration and storage sharing,respectively.In addition,POGs enables private log view to help distinguish which container the logs belong to.The experimental results show that POGs can effectively enhance system log isolation for containers with negligible performance overhead.
基金ZTE Industry-University-Institute Cooperation Funds under Grant No.20200492.
摘要One particular challenge for large‑scale software systems is anomaly detection.System logs are a straightforward and common source of information for anomaly detection.Existing log‑based anomaly detectors are unusable in real‑world industrial systems due to high false‑positive rates.In this paper,we incorporate human feedback to adjust the detection model structure to reduce false positives.We apply our approach to two industrial large‑scale systems.Results have shown that our approach performs much better than state‑of‑the-art works with 50%higher accuracy.Besides,human feedback can reduce more than 70%of false positives and greatly improve detection precision.
基金funded by the Hainan Provincial Natural Science Foundation Project(Grant No.622RC675)the National Natural Science Foundation of China(Grant No.62262019).
摘要As computer data grows exponentially,detecting anomalies within system logs has become increasingly important.Current research on log anomaly detection largely depends on log templates derived from log parsing.Word embedding is utilized to extract information from these templates.However,this method neglects a portion of the content within the logs and confronts the challenge of data imbalance among various log template types after parsing.Currently,specialized research on data imbalance across log template categories remains scarce.A dual-attention-based log anomaly detection model(LogDA),which leveraged data imbalance,was proposed to address these issues in the work.The LogDA model initially utilized a pre-trained model to extract semantic embedding from log templates.Besides,the similarity between embedding was calculated to discern the relationships among the various templates.Then,a Transformer model with a dual-attention mechanism was constructed to capture positional information and global dependencies.Compared to multiple baseline experiments across three public datasets,the proposed approach could improve precision,recall,and F1 scores.
基金supported in part by the National Natural Science Foundation of China under Grant 62272062the Scientific Research Fund of Hunan Provincial Transportation Department(No.202143)the Open Fund ofKey Laboratory of Safety Control of Bridge Engineering,Ministry of Education(Changsha University of Science Technology)under Grant 21KB07.
摘要Log anomaly detection is an important paradigm for system troubleshooting.Existing log anomaly detection based on Long Short-Term Memory(LSTM)networks is time-consuming to handle long sequences.Transformer model is introduced to promote efficiency.However,most existing Transformer-based log anomaly detection methods convert unstructured log messages into structured templates by log parsing,which introduces parsing errors.They only extract simple semantic feature,which ignores other features,and are generally supervised,relying on the amount of labeled data.To overcome the limitations of existing methods,this paper proposes a novel unsupervised log anomaly detection method based on multi-feature(UMFLog).UMFLog includes two sub-models to consider two kinds of features:semantic feature and statistical feature,respectively.UMFLog applies the log original content with detailed parameters instead of templates or template IDs to avoid log parsing errors.In the first sub-model,UMFLog uses Bidirectional Encoder Representations from Transformers(BERT)instead of random initialization to extract effective semantic feature,and an unsupervised hypersphere-based Transformer model to learn compact log sequence representations and obtain anomaly candidates.In the second sub-model,UMFLog exploits a statistical feature-based Variational Autoencoder(VAE)about word occurrence times to identify the final anomaly from anomaly candidates.Extensive experiments and evaluations are conducted on three real public log datasets.The results show that UMFLog significantly improves F1-scores compared to the state-of-the-art(SOTA)methods because of the multi-feature.
摘要In this paper we have developed a data logging and monitoring system, we validated the system by comparing the result from it with the existing one and found that the system performs slightly better than the existing work in the same area. This implies that the data logger and monitoring system is good and can be used to monitor solar energy variables even at the comfort of our homes. We fitted a model to the generated data and found that the meteorological variables considered accounted for 99.88% of the power output in the rainy seasons while 0.12% of the variation was not explained due to other factors. Solar panels inclined at an angle of 5° (Tilt) and facing South Pole perform optimally.
基金supported by ZTE Industry-University-Institute Cooperation Funds under Grant No.HF-CN-202008200001。
摘要Microservices have become popular in enterprises because of their excellent scalability and timely update capabilities.However,while fine-grained modularity and service-orientation decrease the complexity of system development,the complexity of system operation and maintenance has been greatly increased,on the contrary.Multiple types of system failures occur frequently,and it is hard to detect and diagnose failures in time.Furthermore,microservices are updated frequently.Existing anomaly detection models depend on offline training and cannot adapt to the frequent updates of microservices.This paper proposes an anomaly detection approach for microservice systems with multi-source data streams.This approach realizes online model construction and online anomaly detection,and is capable of self-updating and self-adapting.Experimental results show that this approach can correctly identify 78.85%of faults of different types.