Presented by Elastic & Carahsoft

For years, federal data strategies often started with the same assumption: information had to be moved into a central repository before an agency could search it, analyze it or use it to support decisions.

Darryl Peek says that assumption is changing.

Peek, Vice President of Partner Sales for U.S. Public Sector at Elastic, sees agencies moving toward distributed architectures that allow users and applications to reach data where it already exists. Instead of bringing all the data to one place, agencies can increasingly bring search, analytics and AI capabilities to the data.

The shift is driven partly by cost, but it is also about mission effectiveness.

“If you don’t have the right insights and you don’t have the right affiliation with your data and how it’s being stored or referenced, it does limit your overall effectiveness,” Peek said.

Centralized environments can require agencies to transfer enormous volumes of information, pay egress charges and maintain additional infrastructure. A decentralized or cross-cluster data mesh can reduce that movement while giving users access to information stored across cloud, hybrid and on-premises systems.

Peek said agencies can potentially save significant costs while gaining faster access to the insights they need.

The model is especially important as government missions move toward the edge.

The Defense Department has focused heavily on delivering data and computing capabilities to operational users in disconnected or bandwidth-limited environments. Civilian agencies are increasingly encountering similar requirements as their work becomes more mobile and distributed.

Cloud dominated the federal technology conversation several years ago, Peek said, and agencies initially concentrated on moving applications into centralized cloud environments. Today, many are pursuing hybrid approaches and examining how capabilities can operate inside air-gapped or disconnected systems.

Those environments may still need AI, search and data analysis, but they cannot assume a continuous connection to an enterprise cloud platform.

“How do we rethink how we can push AI to the edge versus use it as an interconnected resource?” Peek asked.

That question changes how agencies evaluate both architecture and cost.

Leaders want to understand the price of a solution before adoption, but they also need visibility into expenses that emerge as the platform grows. Legacy security information and event management systems may create substantial egress costs. AI systems may generate unexpected expenses through token consumption, model usage and infrastructure demand.

Peek said those issues need to become part of the conversation early.

Agencies should understand not only how a technology works but also how its economics will change at scale. Discussing those costs in advance allows leaders to make more informed decisions and avoid becoming locked into an environment that is difficult or expensive to expand.

AI itself is entering agencies through more than one path.

Some organizations view it as a specialized tool that can be applied to a specific mission problem. Others encounter AI as an embedded capability inside platforms they already use for search, analytics, cybersecurity or infrastructure management.

Peek sees agencies pursuing both models.

What they share is a need for visibility. Leaders must understand how AI is being used, what data it is accessing and how the underlying infrastructure is performing.

That is where observability becomes important.

Observability allows an organization to monitor the performance and behavior of its technology environment. Once the agency can see what is happening, it can identify problems, investigate changes and respond more quickly.

The value grows when agencies connect IT data with business and mission data.

Security teams may know that a system is generating unusual activity. Program officials may know that a public service is slowing down or producing unexpected results. Bringing those views together can help the agency understand not only that a technical event occurred but also what it means for the mission.

“The riches are in the niches,” Peek said. The value comes from understanding the relevant data points and acting on them in a timely way.

That integrated approach can also reduce the swivel-chair problem that has long affected federal operations. Employees often move manually among multiple dashboards, tools and data sets, attempting to assemble a complete picture from disconnected sources.

Modern data architectures should allow those signals to be correlated automatically.

The need is becoming more urgent as AI transforms cybersecurity.

Continuous monitoring has been part of the federal cyber conversation for years, but Peek believes it is now more important than ever. Agencies must understand what is happening in their environments, identify malicious behavior and get relevant information to the people or systems that can take action.

The volume of activity makes manual analysis increasingly difficult.

Security teams are no longer searching only for a needle in a haystack. Peek cited a description from one of his colleagues: they may be looking for “a needle in a stack of needles.”

AI can help separate meaningful threats from routine activity, but agencies also need the ability to automate an appropriate response. Adversaries are using AI to attack systems, scan for weaknesses and increase the scale of their operations. Government defenders must be able to respond with comparable speed.

“How do we respond in kind with AI?” Peek asked.

That does not mean removing humans from every decision. Automated systems can identify activity, correlate signals and perform lower-risk actions. Higher-impact decisions may still require experienced personnel or a human in the loop.

The same integrated data model supports zero trust.

Agencies need visibility into networks, assets, cloud platforms, applications and devices before they can make effective access decisions. Those elements should not be treated as separate stories. They must be combined into a synchronized view of the environment.

The challenge is that federal technology ecosystems are filled with products that generate information in different formats.

Peek compared the problem to the Tower of Babel. Legacy applications, cloud services, endpoints and security tools may all be speaking different languages.

Open telemetry can provide a common way to collect and understand those signals. Once agencies can bring the information together, they can gain faster insights and respond more effectively across complex environments.

The objective is not to centralize every piece of federal data. It is to connect the information well enough that agencies can find what matters and act on it.

That represents a significant change from the earlier federal cloud model. Instead of building larger repositories and moving everything into them, agencies can create an environment in which data remains distributed but insights move freely.

For missions operating across cloud, on-premises and disconnected systems, that may be the only architecture capable of delivering both the speed and control government requires.