Can the Intel and Defense Community Conquer Data Overload?

Home / Articles / External Non-Government


January 14, 2019 | Originally published by Date Line: January 14 on

The Department of Defense (DoD) and intelligence community is coming to grips with the realization that it might have too much to make sense of after almost two decades of collecting mass quantities of intelligence data. This realization was one of the catalysts for DoD’s flagship machine-learning effort dubbed Project Maven; with the hours of imagery data collected from aerial platforms, maybe algorithms could do the processing and unburden analysts.

However, these algorithms rely on data sets that have been prepared to be analyzed, which is called structured or tagged data. Leaders are now asserting that many of these data sets have not yet been tagged, thus potentially rendering the latest technology advancements useless.

“There’s a lot of unexamined data … that you can apply machine learning to identify objects of interest in the seams,” Dawn Meyerriecks, director of Science and Technology at the Central Intelligence Committee (CIA), said Sept. 4 at the Intelligence and National Security Summit hosted by the Intelligence and National Security Alliance (INSA) and the Armed Forces Communications & Electronics Association (AFCEA).

DoD and the Intelligence Community (IC) have been very good at the data gathering piece, Lt. Gen. John Bansemer, assistant director of National Intelligence for Partner Engagement within the Office of the Director of National Intelligence, said Sept. 5 at the same conference. However, now the community as a whole must focus more on how to enable analysts to make sense of all this data and get to the essence of decision to provide an advantage to commanders.

After all, the data is useless if it cannot be analyzed and made sense of faster than the adversary. Some in the military are beginning to tag data immediately in order to allow for faster analysis by machines.