Measuring and Modeling the Label Dynamics of Online Anti-Malware Engines
报告摘要:VirusTotal provides malware labels from a large set of antimalware engines, and is heavily used by researchers for malware annotation and system evaluation. Since different engines often disagree with each other, researchers have used various methods to aggregate their labels. In this talk, I wil discuss our recent research project on categorizing, reasoning, and validating common labeling methods used by researchers. We first survey 115 academie papers that use VirusTotal, and identify common methodologies. Then we collect the daily snapshots of VirusTotal labels for more than 14,000 files (including a subset of manually verified ground-truth) from 65 VirusTotal engines over a year. Our analysis validates the benefits of threshold-based label aggregation in staillizing fles' labels, and also points out the impact of poorty-chosen thresholds. We show that hand-picked "trusted" engines do not always perform well, and certain groups of engines are strongly correlated and should not be treated independently. Finally, we empirically show certain engines fail to perform in-depth analysis on submitted fles and can easily produce false posives.Based on our findings. we offer suggestions for future usage of Virus Total for data annotation. This work was published in Usenix Security'2020. 报告人简介:Linhai Song is an Assistant Professor at the PennsyIvania State University. Linhai has published more than ten research papers on top-tier conferences, including ASPLOS, PLDI and USENIX Security Linhai won the MICRO' 2014 Best Paper Runner Up and the ACM SIGPLAN Research Highights Award in 2011. Linhai got his Ph.D degree from University of Wisconsin-Madison in 2015. His research interests include systems, programming languages and security.