Scale AI to set the Pentagon’s path for testing and evaluating large language models

Published on February 20, 2024



The company will create a comprehensive T&E framework for generative AI within the Defense Department.

The Pentagon’s Chief Digital and Artificial Intelligence Office (CDAO) tapped Scale AI to produce a trustworthy means for testing and evaluating large language models that can support — and potentially disrupt — military planning and decision-making.

According to a statement the San Francisco-based company shared exclusively with DefenseScoop, the outcomes of this new one-year contract will supply the CDAO with “a framework to deploy AI safely by measuring model performance, offering real-time feedback for warfighters, and creating specialized public sector evaluation sets to test AI models for military support applications, such as organizing the findings from after action reports.”

 

Other news

Houthi Militants Deploy a Drone Sub for the First Time

February 19, 2024
In this 2022 file image, Iran's Islamic Revolutionary Guard Corps unveiled a new model of torpedo-shaped drone submersible. Iran provides advanced weaponry to the Houthi movement (IRGC file image)Since the Houthi rebel faction began attacking Red Sea shipping last fall, U.S. forces have encountered and engaged a wide variety of anti-ship weaponry in and around Yemen. Shoot-downs and on-the-ground

Army picks wearable computers with artificial intelligence (AI) for documenting patients on the battlefield

February 20, 2024
The goal is for the autonomous documentation to work completely without network connection while documenting patients on the battlefield.FORT DETRICK, Md. – U.S. Army battlefield medicine experts needed a soldier-worn computer to enable medics to document patients autonomously through passive sensors and artificial intelligence (AI) algorithms. They found their solution from Tomahawk Robotics in M