Pentagon should experiment with AIs like ChatGPT — but don’t trust them yet: DoD’s ex-AI chiefs

The potential of “generative AI” is too big to ignore, agree retired generals Jack Shanahan and Mike Groen, who both led the Pentagon's Joint Artificial Intelligence Center (JAIC). However, its tendency to “hallucinate” and make up information is “a showstopper” for the moment, making it unfit to be directly incorporated in defense or intelligence at this stage. Nevertheless, the generals agree the DoD should quickly start experimenting with generative AI — with the right amount of caution and careful training for would-be users — so that the Pentagon is prepared to onboard such systems when and if the hallucination problem can be fixed.

 

In a simple exercise, generative AI was asked to produce the biographies of Shanahan and Groen. While the general picture was accurate in the results, there were many incorrect details added. As opposed to earlier software that did memorize specific facts and inferring general patterns, the industry now opted for a type of AI called a transformer, which only encodes the probable correlations between words or other data points. So it selects words not based on facts from the dataset, but rather based on the likelihood that this word is relevant. In other words, it predicts what the next word should be, based on correct data (if available) but also on a lot of other data that is around and that the AI might consider likely to be correlated.