publications
Publications in reverse chronological order.
2026
- KDDOmniVul: A Holistic, Multi-Turn Conversational Benchmark for LLM-Based Vulnerability AssessmentVishnu Teja Kandalam, Viet Duong, Xiaochang Li, Minghui Yin, Vamsi Shankar Simhadri, Hung Pham, Huajie Shao, Xiaokuan Zhang, and Yue XiaoIn Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, Aug 2026
With more than 20,000 Common Vulnerabilities and Exposures (CVEs) reported annually, software vulnerabilities represent a critical cybersecurity challenge. This volume has intensified the demand for automated detection and analysis, motivating the integration of large language models (LLMs) for such tasks. However, existing vulnerability benchmarks are not suitable for evaluating LLMs’ capabilities in vulnerability assessment, as most of them 1) rely on narrow data sources, 2) lack deep context, and 3) focus on single-turn Q&A rather than realistic, multi-stage analyst workflows. To address this gap, we introduce OmniVul, a comprehensive multi-turn benchmark for LLM-based vulnerability assessment. OmniVul comprises 2,000 CVEs with question–answer pairs spanning 23 attributes, including detection, code localization, root cause analysis, and patch suggestion. We employ an automated workflow to aggregate multi-source data via Retrieval-Augmented Generation (RAG), ensuring quality through LLM-as-a-Judge filtering and conformal prediction calibrated by human expert annotations. An evaluation of five state-of-the-art LLMs on OmniVul reveals distinct performance gaps, with top-1 accuracy remaining below 50% on average for vulnerable code detection and CVE identification. Our evaluation also demonstrates that current models lack critical reasoning capabilities for reliable vulnerability assessment. These results highlight the importance of OmniVul for advancing research in evaluating and fine-tuning LLMs for vulnerability assessment.
@inproceedings{kandalam2026omnivul, title = {OmniVul: A Holistic, Multi-Turn Conversational Benchmark for LLM-Based Vulnerability Assessment}, author = {Kandalam, Vishnu Teja and Duong, Viet and Li, Xiaochang and Yin, Minghui and Simhadri, Vamsi Shankar and Pham, Hung and Shao, Huajie and Zhang, Xiaokuan and Xiao, Yue}, booktitle = {Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2}, pages = {9195--9206}, year = {2026}, month = aug, publisher = {ACM}, doi = {10.1145/3770855.3817475}, } - ICRAMulti-Robot Learning-Informed Task Planning Under UncertaintyAbhish Khanal, Abhishek Paudel, Hung Pham, and Gregory J. SteinIn IEEE International Conference on Robotics and Automation (ICRA), 2026
We want a multi-robot team to complete complex tasks in minimum time where the locations of task-relevant objects are not known. Effective task completion requires reasoning over long horizons about the likely locations of task-relevant objects, how individual actions contribute to overall progress, and how to coordinate team efforts. Planning in this setting is extremely challenging: even when task-relevant information is partially known, coordinating which robot performs which action and when is difficult, and uncertainty introduces a multiplicity of possible outcomes for each action, which further complicates long-horizon decision-making and coordination. To address this, we propose a multi-robot planning abstraction that integrates learning to estimate uncertain aspects of the environment with model-based planning for long-horizon coordination. We demonstrate the efficient multi-stage task planning of our approach for 1, 2, and 3 robot teams over competitive baselines in large ProcTHOR household environments. Additionally, we demonstrate the effectiveness of our approach with a team of two LoCoBot mobile robots in real household settings.
@inproceedings{khanal2026multirobot, title = {Multi-Robot Learning-Informed Task Planning Under Uncertainty}, author = {Khanal, Abhish and Paudel, Abhishek and Pham, Hung and Stein, Gregory J.}, booktitle = {IEEE International Conference on Robotics and Automation (ICRA)}, year = {2026}, }