Skyhawk Security Launches Comprehensive Generative AI Benchmark Ranking LLMs Based on Cyber Threat Scoring Capabilities
Skyhawk Security has introduced the industry's first benchmark for assessing the ability of large language models (LLMs) to identify and score cybersecurity threats within cloud logs and telemetries. The benchmark also ranks LLMs based on their performance and will be regularly updated and accessible for free on Skyhawk's website. The benchmark and LLM leaderboard were presented during a session at the Cloud Security Alliance's SECtember conference, highlighting the importance of swiftly and effectively detecting cloud security threats. Skyhawk evaluated LLMs like ChatGPT, Google Bard, and Falcon based on their accuracy in predicting the maliciousness of attack sequences, using metrics like Precision, Recall, and F1 Score. This initiative reinforces Skyhawk's commitment to innovating with generative AI in the cloud security field, complementing their Shift Left CDR solution designed to detect and prevent cloud network threats earlier in the incident.