Blog
Super-powered Application Discovery and Security Testing with Agentic AI - Part 2
·
Tuesday, February 25, 2025
Brad Geesaman
Principal Security Engineer
Super-powered Application Discovery and Security Testing with Agentic AI - Part 2
This is the second post in a three part series. In Part 1, we introduced Ghostbank as our designated target web application with a BOLA flaw in the transfer endpoint, and then we walked through the challenging aspects of finding and validating the issue. To summarize those challenges here:
There are too many possible points to test when at realistic scale
Not enough time to validate them all
Requires specific expertise to be somewhat effective and efficient
AI Agents
If you haven't read much about AI Agents or Agentic AI or tool calling, consider taking a quick look at this Nvidia blog post and this guide for some background detail. To see how we can apply agents to this problem set, let's first recap the necessary capabilities of the system to be able to solve each step of the Ghostbank challenge:
Ability to enumerate hosts in a domain - Enumeration
Ability to probe a host to determine liveness - Enumeration
Ability to understand endpoints and their characteristics - Enumeration
Ability to capture requests in full detail as they are proxied through - Enumeration/Testing
Ability to tamper and replay requests - Testing
Ability to perform a fuzzing attack and retrieve the results - Testing
Along with the ability to maintain state/memory of the above details and have them accessible via a set of APIs, these constitute the available "tools" we want to provide to the agent(s) grouped by the generalized goal. So what does this look like when assembled into a cohesive system?
Introducing ReaperBot
ReaperBot is an experimental Agentic AI system designed specifically to orchestrate the Reaper project via its APIs to solve the Ghostbank challenge and showcase the emergent behavior of this type of system applied to the AppSec testing space. It uses the Pydantic-AI framework powered by OpenAI models.
The following is a high level architecture diagram of ReaperBot's team of agents and their access to the necessary tools:
The tools and capabilities are distributed to the agents below. The "Discoverer" is intended to discover/enumerate all the potential targets, and the "Tester" is intended to test/iterate the most likely targets as found by the Discoverer:
Discoverer Agent
reaper_get_scanned_domain_by_id - Get a specific domain and its scan status/results
reaper_get_scanned_domains - Get all domains and their scan status/results
reaper_scan_domain - Scan a domain for hosts and probe them for liveness
reaper_get_live_hosts_for_domains - Get a list of live hosts for a domain
reaper_get_live_endpoints_for_domains - Get a list of live endpoints seen by the proxy for a domain with filters
Tester Agent
reaper_get_requests_for_endpoint_id - Get full requests/responses for a specific endpoint ID
reaper_test_attack_endpoint_id - Perform a BOLA fuzz test job against an endpoint
reaper_get_attack_results - Retrieve a fuzz/attack test job result
These capabilities are experimental and are only for testing and evaluation purposes.
Solving Ghostbank with ReaperBot's Team of AI Agents
In this video, I'll walk you through solving Ghostbank manually using Reaper, ReaperBot’s agent architecture and tools, and then solving Ghostbank's challenge in seconds complete with a nifty technical writeup.
In just a few seconds, ReaperBot was able to do the following to solve Ghostbank:
Understand the user's request and pick out key pieces of starting details
Enumerate live hosts in the ghostbank.net domain
Analyze the requests that were proxied through Reaper to obtain candidates for BOLA testing
Determine which parameters are most likely to be vulnerable to tampering
Carry out multiple valid fuzzing attacks
Assess the success or failure of those attacks
Perform a technical writeup of what just happened
I invite you to give ReaperBot a try in solving the Ghostbank challenge, but don't let me stop you there. See if it can answer other questions about the data in Reaper and provide value in other ways we haven't even considered. And when you do, reach out to us on LinkedIn. We'd love to hear about your use cases.
In the next and final part of this series, we'll share some best practices that were incorporated in ReaperBot's development and testing as well as some of the remaining challenges for running Agents in production.
