Hiring AI Agents for SOC Teams and Building a Hybrid Workforce

Pyo Kwon
Jun 8
6 min read

PAGO Gartner Security & Risk Management Summit On Site Report

PAGO is attending the 2026 Gartner Security & Risk Management Summit in National Harbor, Maryland, and sharing key insights from the security sessions discussed on site. As the fifth report in this series, we examine Cyber Managers: How to Interview and Hire an AI Agent for Your SOC presented by Eric Ahlm.

*Cyber Managers- How to Interview and Hire an AI Agent for Your SOC by Eric Ahlm*

Executive Summary

Introducing AI into the Security Operations Center is not simply a technology selection exercise. It is a hiring process. This session explored the rapid rise of AI within SOC operations over the past 12 months and provided a practical four step framework for security managers preparing to hire AI agents into operational environments.

The report also presents a vision for the hybrid SOC workforce of 2030, where security professionals and AI agents collaborate by focusing on their respective strengths. Success with AI adoption begins with clearly defined job descriptions, measurable performance objectives, operational guidelines, and well established guardrails. Blind trust is not a strategy. Effective management is. If the previous report provided a glimpse into the structure of the 2030 hybrid SOC, this report focuses on a more practical question:

How should organizations interview, evaluate, and hire their first AI teammate today?

A Shift in How Organizations Think About AI

Eric Ahlm opened the session by emphasizing that AI has the potential to influence every aspect of the security workforce. The discussion focused on how organizations structure teams, define roles, and divide responsibilities between people and technology. Interest in AI powered SOC operations has grown dramatically over the past year. However, security leaders must separate market excitement from operational reality.

While fully autonomous security operations may eventually become possible, Ahlm cautioned against viewing that as the near term future. AI teammates should first be deployed in areas where activities are clearly defined, outcomes can be measured, and operational improvements can be validated against an established baseline. The discussion repeatedly returned to a simple principle: AI should be hired to improve operations.

The Only Metric That Matters: Operational Improvement

According to Ahlm, the purpose of hiring an AI agent is operational improvement. If an AI solution cannot produce measurable gains against an existing baseline, organizations should reconsider the investment. Several examples were presented during the session:

Evaluation Area	Example Improvement
Efficiency	A task previously requiring two hours is completed in five minutes
Error rate	Alert triage error rates drop from 20% to 5%
Performance improvement	Mean Time to Detect (MTTD) decreases from two hours to thirty minutes
Skills required	Junior analysts perform work previously limited to senior analysts
Program growth	Investigation coverage expands from 85% of incoming telemetry to 100%

A 4 Step Guide to Hiring AI Agents

Step 1: Define the Job Requirements

Organizations should begin by identifying existing operational activities that consume significant time and resources. The most suitable early AI use cases are not entirely new functions. They are existing tasks already performed by security teams today. Ahlm emphasized that successful adoption usually begins in areas with substantial workload volume and well understood processes.

Examples discussed during the session included alert triage, augmented investigations, alert enrichment, reporting. The more clearly a task can be defined, the easier it becomes to evaluate AI performance.

Step 2: Establish Performance Objectives

Organizations should document expected outcomes using measurable data rather than subjective expectations. Ahlm recommended defining performance goals before evaluating technologies. Security leaders should understand exactly how operational challenges will be improved and how success will be measured.

Use Case	Primary Benefit	Current Human Workload	Desired AI Outcome	Expected Business Impact
Alert Triage	Efficiency	3 Tier 1 analysts (120 hours per week)	AI handles 80% of triage activities with only 10 hours of weekly oversight	Measurable workload reduction
Augmented Investigations	Performance Improvement	Tier 2 analysts spend approximately 2 hours per investigation	AI enriches data and generates investigative hypotheses, allowing analyst validation within 30 minutes	MTTD reduced by 1.5 hours
Augmented Investigations	Skills Enablement	Only senior analysts can perform investigations due to tool complexity and query expertise	AI natural language interface handles tool syntax and query generation	Junior analysts can perform more advanced investigative tasks

Step 3: Source AI Talent

Organizations can acquire AI capabilities through one of the following four approaches, depending on their operational environment and internal maturity.

The third option, engaging a service provider, was specifically discussed during the session. Reflecting on this point, PAGO recognized a strong connection to the concept of validation that has been part of our MDR service approach since 2017 when delivering AI based security solutions.

The application of AI to areas such as alert triage, as discussed by Pete Shoard and Eric Ahlm, can significantly improve operational efficiency. However, the need for validation does not disappear. AI can assist with analysis, prioritization, and investigation, but the responsibility for verifying outcomes and ensuring appropriate actions remains essential.

For this reason, organizations should carefully evaluate MDR service providers that have experience operating in AI assisted environments and can effectively adapt to, manage, and lead this transition.

Purchase a Dedicated AI SOC Solution: Adopt a specialized AI security operations platform available on the market.
Leverage Existing Platforms (Use Existing): Utilize the AI capabilities and roadmaps of existing SIEM, ticketing, and security management platforms.
Engage a Service Provider (Engage MDR): Utilize hybrid services delivered by a managed security service provider.
Build In House (Build In House): Develop internal AI capabilities by leveraging proprietary data and existing organizational AI initiatives.

Step 4: Conduct the Interview and Define Guardrails

When evaluating AI solutions, organizations should move beyond feature comparisons and adopt the mindset of a hiring manager.

Ahlm suggested asking questions similar to those used during employee interviews:

Can it perform the job?
Has it done this work before?
Can it demonstrate proven results?
Does it have references, evidence, or measurable outcomes?

The discussion repeatedly returned to data, metrics, and measurable improvement.

Organizations should also evaluate:

Whether performance metrics align with business objectives
How quickly value can be achieved after deployment
How disagreements between AI recommendations and human judgment will be resolved

One of Ahlm's favorite interview questions focused on a different topic entirely: What are the grounds for termination? Organizations should establish clear guardrails defining which actions, policy violations, or unauthorized data access scenarios immediately stop AI operations. Successful AI adoption requires clearly defined boundaries as much as it requires technical capability.

The Hybrid SOC Workforce of 2030

The session described the 2030 SOC as a hybrid workforce rather than a fundamentally redesigned organizational structure. Instead of dramatically changing team structures, each security professional is expected to work alongside their own AI teammate.

In this model, AI handles repetitive tactical execution, while security professionals focus on setting direction, managing programs, exercising judgment, and making operational decisions. Responsibilities are divided according to the strengths of each, allowing AI and human expertise to work together within the same security operation.

Human vs. Agentic Responsibilities

Role	Human Responsibilities (Strategy / Management)	AI Responsibilities (Execution / Tactical)
Security Operations Manager (SecOps Manager)	• Performance reporting and team coordination • Program growth and strategic direction	• Incident report summarization and GRC documentation • Operational insights, including error rate and performance oversight
Detection Engineer	• Detection strategy and development • Prompt design	• Detection code recommendations and generation • Code testing and troubleshooting
Systems Engineer	• System efficiency management • API / MCP integration planning	• System analysis and processing • Command syntax normalization
Threat Analyst	• Incident handling • Response coordination	• Initial alert triage and enrichment • Augmented investigations and response playbook generation
Threat Expert	• Threat hunting leadership • Offensive security program planning	• Threat hunting hypothesis generation through historical analysis and indicator collection
Exposure Management Specialist	• CTEM coordination • Organizational mobilization, communication, and escalation management	• Augmented validation • Automated exposure remediation
Offensive Tester	• Offensive testing coordination and scope definition	• Penetration testing support • Secure code review

Conclusion

As the session concluded, Eric Ahlm summarized the core message with a single statement:

"Building an agentic workforce for security operations starts with defining job roles, metrics, and hiring requirements."

Rather than beginning with a discussion about AI platforms, organizations should begin by understanding the work itself. Defining responsibilities, establishing baselines, measuring outcomes, and setting operational guardrails provide a more realistic foundation for AI adoption.

As an MDR service provider, PAGO also found the discussion around alert triage particularly relevant.

The conversation should not begin with the assumption that an AI SecOps platform can simply replace tasks currently performed by analysts. A more practical starting point is understanding the existing baseline. How long does a Threat Analyst spend handling malware alerts? How much effort is required to complete investigations? Which activities consume the greatest amount of operational time?

Once those baselines are established, organizations can evaluate where AI teammates provide measurable value. The word "hallucination" is frequently discussed in today's AI conversations. At the same time, people often develop unrealistic expectations about what AI can accomplish. This session provided a practical framework that helps security leaders approach AI adoption with measurable objectives, realistic expectations, and operational discipline.