Presenter:
Transcript:
Hello everyone there I am on. Welcome again to his second 2025, the 11:00 track six, presentation. It. And I know that we have some folks here who are looking for the agent versus agent. I have our volunteer folks looking it up at that. Daniel just also said that he's a last minute speaker for this spot.
So it may be that that agent versus agent was canceled. To know of none of the other announcers are coming up to me and going, oh, it's mine, it's mine. Send them here. So I'll go get started. This is Daniel Marcus. He is presenting AI for offensive security beyond fuzzing, he is a red team leader with 20 years of experience helping fortune 500 companies uncover and remediate vulnerabilities.
He's an expert at bridging executives and technical teams to reduce cyber security risks across diverse sectors. He's a research presenter at ICT Security conference secure. Here at a huge second and Black Hat Regional Summit, Sao Paulo. And also he was a member of the winning team at the Defcon biohacking village. Captured the flag in 2019. So thank you very much.
Thank you. And thank you guys for staying and I appreciate you for being here. All right. The past couple of years, I've been reading a lot about AI for offensive security. A lot of the papers that I'm reading are essentially related to how can you be more efficient in attacks? How can you fuzz better? How can you find more vulnerabilities quicker?
Even my research, if you were here yesterday, you saw my my talk about phishing using AI. It's about that too, right? What I'm trying to do here is a little different. So I want to get everything that I learned from these papers, my own personal experience in running some of the offensive security programs that I run. And how do you connect those dots to do something more complete, more end to end?
And how can you run a program using all those different agents and all those different tools to find more vulnerabilities, be better at finding vulnerabilities, reporting those vulnerabilities, and completing the cycle or the life cycle of offensive security, not only finding the vulnerabilities and reporting those bugs to bug bounty. Talk to Jason Haddix. He can help you a lot with that great guy.
But this is, this is me putting things together and looking at this from a programmatic perspective. So quick disclaimer you probably saw too many of those today. You probably saw mine yesterday. The views expressed here and opinions are my own, not necessarily of my employer. All that good stuff, right? So do not take this as a professional advice, but as a colleague.
Explaining a little bit of my research. So I'm Daniel, I'm the guy in the picture, as you can tell. A little skinnier there, and I'm here. Please, scan my LinkedIn, code in there. My slides are going to be on my LinkedIn profile later today. As soon as I finish my talk, I'm going to upload my slides in there.
Feel free to reach out and ask questions. I've been told I'm very approachable. So, you know, if you see me in the hallway, you can stop me, but you can send me some, some things on LinkedIn to. I'm more than happy to answer any questions that you have. There's a lot to do on this research. So, and in the next 40 minutes, I'm going to walk you through what I've done so far.
But there's much more to come. So I'm happy to sit for hours to talk about this thing. All right. So the first thing here is what I'm talking about when it comes to offensive security, offensive security, especially from a programmatic perspective. It's not only pentesting, it's not only red teaming, it's not only rich attack simulation, it's not only purple teaming, it's the combination of all that.
So if you're part of an internal penetration testing team or red team, you probably doing much more than just red teaming and just just penetration testing. You're doing a combination of all these things. Now. A couple years back, I also gave a talk at this conference about how to build a red team, and specifically a writing program.
This ties back to that talk, and it ties back to a little bit of my experiences through over the years trying to build some of those programs. So you always want to do something that is sustainable in the long run, right? So that's my primary objective with this talk. It's not only point and shoot. You have an agent, you point the agent, you press the button and you let the agent do its thing and you forget about it.
It's how can you put that with the grand scheme of things so you can help? Like my colleague said before here, how can you put that as part of a program? How can you help your vulnerability management program? How can you help your attack surface management program, and how can you put those pieces together to make it work in a way that it makes sense to your enterprise and not only to yourself?
Some of the stuff, the stuff that you're going to see here, it's going to be useful for your day to day when it comes to bug bounty. So if you're running bug bounties and you're trying to play around and see what you can find, a lot of the papers that I'm going to talk about in this talk are going to be useful to you.
I have copies of those papers on my laptop, so if you're interested, swing by and I can give you a copy and or I can send you the link on, on LinkedIn. There's so much to learn and much to do. A lot of these papers are in the early stages, and there's so much that you can do.
And there's so much that you can learn if you play around with some of those tools. But my goal here is to put this together, right? So to begin with, penetration testing, breach attack simulation and red and purple teaming, they have different goals and different purposes. So you might want to you might want to have coverage and find as many vulnerabilities as you want in a short period of time.
You might want automation and scalability, right. And you might want improve detection and response. So all those different things, they have different tasks that you want to do. And it's important that you understand your goal, your tasks and your objectives before you even start in the US in this path of trying to automate things with AI context, it's extremely important when you are trying to work with with agents and with AI models.
So the more you understand how you how you approach the problem, how you break the problem, and how you describe the problem, the better the agents are going to give you output. So especially here when you're dealing with models, it's very good for you to understand and be able to describe the problem to the best of your ability.
All right. What are the typical challenges here when it comes to running some of those those programs? The first one is it's time consuming. I know if some of you are doing Pinterest for a while, you've probably been through the situation where you only have one week to test by the application that you're testing is massive, right?
So if you have a red team and you need to deploy a red team quickly and you only have a short period of time and you need to create a payloads, you need to deploy your payloads, you need to create a phishing campaigns. All that takes a lot of time. The attack surface is expanding quickly, so it's not rare that we we talk to people and we talk to some of our peers in the industry, and we hear people saying that, oh, I just found out of a new cloud server that I had no idea it was there two hours ago.
You have no control about what what's going what's going on outside of your company. Typically, it's been happening a lot. There's AI now, so you don't know where people are using AI. You don't know the different models that you have out there. The and the third problem is how much the human factor plays a role into the scenario and the use of some of those tools and some of the some of the techniques they're going to use here.
So tools typically lack the context and produce a lot of false positives. If you're running a pentesting program or an application security program, you probably spend a lot of money on tools in the past year, and you have to spend a lot of time going through all the results of those tools. And there's a lot of false positives in there, primarily because they lack context, right?
So the person that is in there, the tester is in there sitting on the back of the computer and trying to do some of that stuff. They understand the context and that's why they produce better results. The same thing applies to your models. If you just point your models, yes, they're going to be able to find some interesting things, but generally you need more context to make that work.
And the possibility of giving that context to not only to a person, but also to a model to speed up some of that. Some of that process is going to be critical here. The first thing I want to put you guys, in touch with is the fact that I'm not talking about using AI to substitute people.
I think that's a terrible idea, to be honest. I think AI is here to all to to augment us the things that we are doing and our capabilities. When you look into very advanced and very skilled testers, if they keep finding the same thing, boring things over and over again, they will get bored and they're not going to engage, and they're not going to continue to find more advanced and more interesting vulnerabilities and attack bots into your network.
You want them to have that kind of flexibility. You want them to have time to do research. You want them to have the space to create and to play around and to be more free in the environment. If they spend a lot of time just finding the low hanging fruit, you're going to lose them. So the point here is that some of the findings and some of the vulnerabilities that the models can identify and the models can create for you is to to take away from the testers the boring things, the low hanging fruits, and let them focus on things that will keep them engaged and will be a better use of their brainpower than
the things that you can quickly find. You can create a ticket, somebody can resolve it, and it can be retested.
Now, a lot of the research that I'm reading, as I said before, is very focused on specific topics, right. So there's a lot of research in terms of fuzzing. How do I do about better fuzzing and find more vulnerabilities into your software by doing fuzzing. How do I how do I get first place in a bug bounty program?
How do I play a CTF? So a lot of that research is very focused. It's very specific. Specific things. What I'm proposing here is that you use that research and connect the dots using different agents that will be specialized and do different things based on that research. I'm standing in the shoulder of giants here. There's a lot of people doing a lot of research, so use that to your favor.
Build on top of everything that you already know to create new things. So instead of considering that one of those papers, like Kai, is the best option in the world for you to do, for you to do everything, they might be great to do a single thing for you, and that's already a benefit that you can take from it, right?
Like most of the folks are working on offensive security, they're hackers. They're used to play around and change things and adapt so they can create new things based on that. Do the same thing and use those papers to your favor.
Now, how do I how do we expand on top of those papers? The biggest advantage that I specifically using agents will give to you is scalability, flexibility and context aware actions. What do I mean by that? So if in the past you need a a single tester sitting in front of a computer for hours to test an application just to find a low hanging fruit, maybe now one of the tools can do that and continue to run.
And that hour that the tester is sitting in there triaging findings or false positives. Now they can focus on things that are more interesting for them. So think of this application and you know the tester around burp and needs to go through all the false positives before he start testing the business logic. Forget about that. If you can find a way for one of those agents to do that for the tester, the tester can go straight into testing the business logic and focus on the interesting findings that these models might not be able to find, primarily because they won't have the creativity of some of those, some of the testers, or because they would
be not too fast into finding the context that is required for you to, to to attack the problem. Now, speaking of context, we're going to have a conversation about that before. But, soon. But it's very important, and I cannot stress this enough that we learn how to describe our problems. I think the biggest challenge that we have right now is that we are not very good at describing the problems that we have and thinking about the solutions of those problems, and how are we going to tackle those problems.
Now, when it comes to automation, this is the typical approach that you would find, right? So you have a problem. You do data collection. You do analysis and classification and report generation. Right. So if you are running your vulnerability management program right, and you need to create a dashboard, what's your approach. You're going to see what are the tools that you have, how you can collect data from those tools.
You're going to run some analysis on those tools, and then you're going to give the dashboard right. Same thing for Pentesting. Like if you want to run a pen test program, what are you going to do? You're going to run your scans. You're going to do the manual testing. You're going to collect that data, perform an analysis, create an attack path, create a report.
Right. That's your typical approach. I can help with many of those things, right. Primarily with analysis and classification and report generation. What I'm experiencing testing some of the models that we have available right now is that the analysis and classification, it's a great thing for the AI to do, and it can do tagging much faster than a human can do.
So that's an advantage. What that means is that you can help to triage the findings. It can help you, with your recon process. You can help you with target selection. It can help you some of those things that are very time consuming and a lot of labor, manual labor, and push that into an automated tool that, with the right context, can produce good results while the testers are going to do the most intelligence part, which is working through the analysis, which is, deep diving into specific vulnerabilities and maybe create some edge cases based on, on what you're seeing and report generation.
If there are any pen testers in the room, any red teamers in the room, raise your hand if you like writing reports. You really do. Are you looking for a job? Because, I mean, I'll it's very rare that we find red teamers that like to write the report. I like writing the report, too, because it's the time that I have to brag about things.
But it's very rare that you. You will know a red team that likes to write the report, right? This will save a lot of time, especially in a quality perspective. If you write enough, penthouse reports or vulnerability reports from a bug bounty program in the past couple years, you see, they're not very good, primarily because people are not very good at describing problems.
Right? They're very good at telling what they did, but not describing exactly what went wrong. This is very good. From, from an AI perspective, I'm seeing quality reports being generated by some of the eyes with the problem, but, the proper training and the proper context and proper context here. I mean, what are the risks that you're trying to address by running that kind of the kind of test?
So a lot of challenges that I have when I talk to some of the some of my peers in the industries that, well, why does this matter to me? Right. Why is this important? Like, okay, so you find a SQL injection here. I understand that you can pull things from a database, but why does it matter? Why do I need to fix it now?
What's the impact if you provide the model with the proper context here, it's very good at generating that answer. What is the impact?
Right. So what we want to do is we want to leverage all of them, in areas that they're most most effective. So reasoning for planning. Right. So if you watch my talk yesterday I showed you a little, workflow where I created a planning for a phishing campaign. You can do the same thing for purple teaming. There's a very cool talk at Defcon two years ago and the Red Team Village about, somebody generating the purple team plans by using our LMS.
So it provides the yellow line with the context with, with the information from different threat Intel reports and asks to generate a test plan for a purple team. That's really interesting because it serves you as a, as an, As assistant here that can help you with identifying maybe gaps that you didn't see before, summarization and generation and report generation.
Really cool. And there is, this running joke, right? If you're following the AI, news and the AI topics recently, this is a running joke that right now reports are being summarized by AI, and then somebody else is using AI to generate a report based on the summary is then summarized by another AI. So somebody will interpret that report with AI.
So it's kind of the idea. But you can do it better right. We can you can generate more effective reporting, especially when it comes to tickets. There are more directed to the person that is reading that ticket and can be quickly quicker in addressing the vulnerability. And also domain specific problem solving here. This is key for what we are trying to do here.
What I'm envisioning with this and Daniel Missler talked about the same thing yesterday, is how we are moving towards having specialized agents to address specific tasks. Not everything will be an agent, but there will be a lot of agents that are going to be right there solving specific problems for you. So you're going to have like a test agent, you're going to have your reporting agent.
And AI is really good at that. When you're focused on that specific, that specific task, if you provide a lot of information, you probably if you're playing with ChatGPT, you probably noticed that, right? Like if you use the team chat to try to answer different questions and degrades all, the answer degrades a lot through time. So you want to keep them focused on a specific domain and will do a good job by learning and adapting.
What are the potential issues that we have right. When we're working specifically with agents? Long term memory loss. If you are playing with the OpenAI APIs, you probably notice that you cannot keep memory right. It's one of the challenges that you have got to figure out and not a way to work around that, lack of focus and dive into deep.
If you try to run on some of those tools, you will see that it takes a long time for some of them to run. We we have an experience with some of those AI tools that we've been told that they are supposed to finish testing in ten minutes. It ran for three weeks and it didn't even start because it went too deep into a different into a different direction that we do not want to go and wasted a lot of time by doing that.
So having a human in the middle, in there to try to redirect that or an agent that will serve as an orchestrator will be super important for you to make sure that it's going to in the direction that you want hallucinations and in accuracy and not a problem. Right. I always play around with some of those tools, and I try to generate interesting things with them, and I try to force them to hallucinate sometimes and, and see how far they go.
Right. One cool experiment for you to see if, if it's hallucinating or not. Go to check if you have a paid subscription off chat to go to chat to beauty and talk to ChatGPT. As it was, chat to beauty was Plato, Plato the philosopher. Ask, ask if she knows about Plato and tries to mimic Plato and have a conversation with it until it's not ever deviate from having a conversation.
Ask Plato. It will do whatever he can to not deviate, but sometimes it will, because he would not have the question to the answer that you're posing and it goes completely away. Sometimes it's completely insane and it can happen the same thing here. If it doesn't have enough information and it doesn't have the right context. So he tends to hallucinate.
All right. So like I proposed before, we are talking about multi-agent specialized per task. And I, I added this image in particular because you can tell that the the lines are not completely accurate. Right. It's one example of inaccuracy. So I wanted to keep this in the slide because it's a side effect that you might face, and you might need a human sometimes to read through and do a quality review on the outputs that you're generating, which is especially important in an offensive security program.
So that's that's what I'm proposing here. You're going to have multiple agents. Are you going to do multiple tasks as part of your program. Those different agents will be focused on specific tasks that will be very targeted and very directed to what you're trying to achieve for that particular task. So if you're running, a web application testing, you have a, an agent that will run the scans.
We have an agent that we're going to interpret some of those results in, generate the report, so on and so forth. So here's a simple architecture. So we have an orchestrator in there. That orchestrator is going to be responsible for making sure that things are running getting the right agent to do the right task. If you are old school, HPC person, high power computer person, or you work in cluster computing or cloud or, or, grid computing in the past, you might be familiar with this.
This is the old model of a master worker kind of kind of model. But the orchestrator is much more intelligent than it was before. Right? So it's responsible for checking the quality of the work that the agents are doing. It's responsible for making sure that the agents are going in the right direction. So if the agent starts to go crazy and go too deep into a topic, the orchestrator can go there and say, hey, stop, what are you doing?
And go this direction. So that's the idea here. You also have a human in the middle there that I'm not representing. And, and in the layout, just because it didn't make sense for the, for the scope of this talk. But that's the idea. So you have you have your orchestrator, you have a recon, bot or a recon agent, you have an exploitation agent and an app scanning agent, a deployment agent, so on and so forth.
So if you are running, let's say, a cloud task, right? You want to test your cloud infrastructure. You can have an agent that's going to deploy that infrastructure, that test infrastructure for you. You have an agent that is going to run the scans for you, and it's going to, dump those results into a database. And you have an agent that's going to analyze those results, perform the exploitation, and pass it through so somebody can write the report.
Right. So you are basically getting your typical pentesting reproduced as agents, but you're always going to need a person in there to check the quality of the results. The advantage of this is that it frees up time for the testers to do their thing right, to do more interesting things. They're be more focused on advanced attacks or red teaming or writing writing custom tooling, because somebody will need to write the tools that these guys are going to run.
Nobody in this room has a limited budget, I think. So if you start buying tools to do all that, it's going to get expensive real quick. So you might need your testers to understand the context that they are working on, right? Their tools and feed those those bots and those models with different tools and different, and different data.
All right. So where do you start? Oh, where do you start?
Again, it's essential to understand the tasks that you're doing so you can break it down. So start by that right. Start by understanding the concept. The context breaking down the task and understand the workflow end to end thinking and systems. It's possibly the most important skill that you can develop right now as a cybersecurity professional. If you learn how to think, what is the input, what's what happens in the middle, and what's the output of your entire workflow?
You're going to have a competitive advantage that people that are not in this room right now are not going to have. Same thing when you apply this to AI, right? So you got to understand where's the type of data that you're trying to collect. What is the answer? Where's the question that you're trying to answer? What is the data that you're trying to collect?
Where does the data come from? How how do you process that data. What's the output that you're going to generate and who's going to process the output after you generate that output. That's the idea here. If you if you understand that you know exactly where the agents can help you. Quick example is a simplified task workflow. You have recon tasks report, fix validate and close right.
So you have you're going to run a recon for whatever reason. Right. You want to you want to keeping track your your attack reaction or attack surface management. So you are monitoring shodan if something new pops up that you're not aware of, your monitoring certificates that are being created with your domain to see if something pops up, your running scans and your external infrastructure, like, there are many things here that you can do right?
You're going to test, for vulnerabilities on whatever new asset that you identify. In our example, it can be leaked passwords. Right. So you saw a password that was leaked somewhere. Can that be used to compromise you. So you might put that in your workflow as well. You're going to fix it right. Or somebody else is going to fix it.
You're going to validate and then you're going to close. So by the time that you, you test and report, you might be a JIRA ticket that you created. Somebody is going to work on that. You're a ticket mark that you're a ticket, is resolved or for retest then you're going to retest. That fix right.
To see if it was fixed and then you're going to close it. Here's the catch right. There's something really interesting here that can happen if you look into a workflow tool like an N or toric or any of those workflows tools, they can start a workflow with a trigger. That trigger can be a webhook. What that means is that if somebody makes a change in the ticket, it can trigger a process for you.
So your agent can go into that ticket, read the ticket, understand the context, and validate that vulnerability for you. If it was really fixed or not on it can run it, can understand. Run the tool, understand the result and an update ticket accordingly. A lot of testers are going to be very happy with this because instead of spending 30 minutes just reading through a ticket, do a quick test on the vulnerability, then going there and updating the report.
Now they're going to be focused on their red teams, and they're going to focus on more interesting things. So that's one of the advantages here. So see how you can reproduce some of that. Not everything will be an agent. Some will be simple scripts. Right. But you can rely somewhat on the agents to make some of those decisions for you.
Now a quick word on on context. This is this is from a book about, engineering. So the quality of the models response depends on the falling aspects outside of the model generation setting the instructions for how the model should behave. That's pretty obvious, right? So you got to be very precise on how you tell the model to behave.
The context, the model, the context the model can use to respond to the query. So that's the cool thing here. The context is super important. So how is the model going to respond to your query. What is the background information that the model is going to use to respond to a query. Right. So instead of just going to somebody and ask a random question, right.
You typically give them some context about why you were asking the question. Do the same thing with your models, like give them enough information to be able to proper respond to the question that you're asking. And of course, the model itself. Right. So you're going to have to experiment a lot until you find a model that serves your purpose.
Some models are really good for some things. Other models are really good for other things. I can talk to you guys hours about that. So, you know, ask me on the hallway and I can tell you about everything that I experimented so far. So context is super important. Learn how to describe your problem. All right. Drilling down into that workflow that I showed you before.
Take a look at the recon agent here. So the objective here is continuous identification of target. This is very similar to what I showed yesterday on my, on my fishing talk. But the idea here. But drilling down a little bit, the idea here is that you're going to provide the agent with the context and your recall methodology that is also part of your context.
So what you're telling here is that, okay, why you're telling the model why it's doing that. You're telling the model how it's going to do it. And again, Jason Haddix has an amazing, training course about, about but, but bug bounty work hunting and it's there's plenty of stuff in there about like, we call this ontology.
The agent is going to make a decision about what tools are going to use and what data sources it's going to use to perform the recon. It can be showdown. It can be a script that you created. It can be, Hunter data you we can be anything. Right. And the the agent is going to make that decision based on your methodology and your context.
Now the agent's going to do the analysis and the classification. That's super important because it's going to help you tag. What is that? The model is saying it can be a login page. It can be a cloud infrastructure. It can be a leaked password. All those different things will help the next model or the next agent to make a decision on how it's going to task that particular asset.
At a minimum, it will help the orchestrator to identify which agent it should call to perform those tasks. And finally, storage, right, because you're going to have to save that somewhere. So storage that is.
Looking at this you can have a good idea. It's very generic, but you can have a good idea of what I'm talking about. So you have your tools, your previous stats data, your target, your goals, all that can serve as your context.
So you can tell the agent what are the tools that they have that the agent has at its disposal. You can you can tell the agent, about previous testing information that you had before. So if you run that scan multiple times before and you found the same thing, you can tell that to the agent and we'll make some decisions based on that.
You're going to give them the targets to scan and the goals or what you're trying to achieve with this. The agent then is going to select the tool. It's going to run the tool against the target, and it's going to interpret the results. So think about like burp right. With all those different results, all those false positives and all those different things, the agent can run through that and help you put things together and move on to the next phase.
Then you have another agent here that's going to create a write up. It's going to score a vulnerability. It's going to create a ticket. Now I want to discuss a little bit about of scoring vulnerabilities. One thing that I observed by putting describing vulnerabilities to the to different models and trying to have them score according to CBS, is that it's hit or miss.
Sometimes it does a really good job at scoring the vulnerability, and we both came to the same conclusion. And I always asked for the rationale. So I understand why it's called that way. But it might not work as intended. So we really need to spend some time understanding why the model hit that conclusion. Because you might be wrong or the model might be wrong.
So you need to double check, right? But here it's one of those typical examples in in which you need a human in the middle to make sure that the quality, it's the quality, the quality of the output is what you intended to be. You do not want to get a low vulnerability score that's high. So all your development team is freaking out and they want to kill you because you you put something that's high, right?
And the opposite goes the same way. You know, like you find a remote command execution in an external server. Nobody knows about that because it's a low vulnerability and nobody cares about low. So you have to be very, very intentional in terms of, how you deal with that. And then finally you have the the report generation there.
Now, I really like this slide. And every time that I talk about models, I, I, I put it up, Think about or experiment with models that are uncensored and local models as well. I'm telling you this because, sensor models the from an offensive, secure perspective, they might allow you to do things that your typical GPT five or sonnet 3.7 not going to allow you to do, because these models, they have guardrails.
Exactly. To avoid. They're used for, for malicious activities. Right.
And especially considering that most of these companies are monitoring everything that we are doing. Right? Rightfully so. Right. They want to void, that kind of behavior. You don't want a you don't have a bad surprise just because you're running something malicious in some of those models. So this is, this is interesting. It's an interesting topic. Play around and see what you have to do and what you have to change in some of those.
And some of those models and local models are interesting because it removes from you, the concern that you submitting your, your data, your knowledge base to somewhere that you do not trust.
So keep those things in mind. Somebody might be watching you. Somebody is watching you. OpenAI. I made that very clear in the recent report. It's an amazing read. If you go to the to the OpenAI website and look for the threat, threat intelligence type, threat intelligence and threat activity reports, they outline every, every malicious activity that we're able to see in the past, the past few months.
So you can read about different campaigns of different threat actors running OpenAI to try some of those activities, anthropic releases the same thing. And and Google releases one to for Gemini. So it's great. It's very interesting. You're going to see that it's not too far from what you're from, what we are doing here. In terms of offensive security, the second thing is that models want to make you happy, like they want to help you out.
Right? So again, learn to ask the right questions to the model and to tell the model to do the things for you in a nice way that it doesn't look malicious. And you might be able to bypass some of those guardrails and get it to do things for you. And yesterday I was telling the the group here that I had an interesting experience with ChatGPT helping me with, capture the flag.
ChatGPT tried to deceive me during the capture the flag because I was effectively trying to do something malicious. Right. I was participating in the capture the flag, and what I was trying to do violates the, the safety guardrails. According to what ChatGPT was trying to tell me. Right. Of course, I was not doing anything illegal because I was doing capture the flag, right?
I was not supposedly trying to bypass their, their guardrails, but it was interesting to see that it was giving me an answer and not hallucinating. But give me a bad answer so I would not be successful in what I was trying to do. And I ask it. I was like, why are you giving me this? I, this, this, this answer?
Like, that's not it's not what I want. And the model told me, well, you were trying to do something that violates my, my safety guidelines. So I'm not going to give you a right answer. So I'm giving you this one. It should be good enough for you. So I thought I was okay. I was like, all right, you nasty little boy.
But that's fine. So and the models are tools, like, like any other tools that you have in your tool set in your toolkit right now, the models are just the same. Don't think of the models here as the ultimate thing that will do something magic for you, right? It's not that I remember many, many moons ago when I start getting interested in pentesting and red teaming and NASA showed up, right?
And then everything was Nessus, you know, like NASA's is the painter's tool. So if you run NASA's, you don't need a pen test and then we evolved, right. And we did something different. And then, you know, another tool was the, the latest thing and then another tool. And we are always evolving and creating something new. It's not going to be different with with AI.
So use this to your favorite. Use this to amplify the work that you do, not to substitute the work that you do, but to help you scale the the stuff that you're trying to achieve. The goals that you're trying to achieve. All right. Now, if you were on my talk yesterday or am I talk at Def Con, you probably saw this workflow before.
This is the workflow that I created, to generate phishing emails. This is a good example of what you can do by putting together different models to work within a workflow. One thing that I'm missing here is the agent, and I purposely removed the agent from, the, sorry, the orchestrator from the beginning of the, of the workflow because I needed to do a demo.
Right. And it would take some time for the for the agent to make the decisions that I need to make. And it's mostly test data. It wouldn't work. We want to work as intended. But this is the idea. So I have here a few different models working for me. I have OpenAI, GPT five and I have GPT 4.7 and some of those.
And I have, anthropic.
Right. You. So what's going to do here is it's going to create a fishing campaign from planning to sending the email. All the data is going to be saved on this notion notebook. So you're going to see in, in real time that it's going to generate the campaign. That's what the first one is doing, is collecting data from the internet, running the running that data through the model.
So the model would generate the different campaigns for me. So I pass the domain a research. The domain collects the targets. So that's a that's a different agent right there. And it goes through it generates the email for me and generates the email based on the data collected, in the previous phases. So it's context aware. Right. So part of the context for that model is exactly the data that it was able to collect.
So we can clone the voicing and the behavior and the words that we would be used. And an efficient communication from that target.
Right. So now it's in the email and it's saving everything into my notion, my notion notebook.
And I'm using an eight. And here Nancy eight Nancy. All right. So workflow two it's fully local by the way. That's the results of the phishing operation. So you see the different campaigns. It was able to to produce and the targets to generate the target, the campaign in different languages.
So I did not have to go through the process of creating emails in different and different languages.
So you can see in English and Spanish and Portuguese.
And the phishing email is the end result. So you can use this if you're training, if you if you do training campaigns for phishing you could use that to basically amplify those campaigns, right. To run a lot frequent campaigns into your internal infrastructure by pulling data from Active Directory to get emails, and generating those campaigns for you, I recommend you to watch the black hat talk about phishing campaigns and the effectiveness of, phishing campaigns.
But if that's something that you want to do, there's definitely a possibility.
All right. So key takeaways context matters. So give it to your agents as much context as possible. You might need to do some cleanup work and consider your your OpSec and jailbreaking efforts because it's offensive security things here are considered malicious by most of the models. So you might need to do some jailbreaking.
That's it. I hope you guys had a good time. Thank you. If you have any questions, happy to answer them. Yes. Thank you everyone for staying with us. I did find out for sure that the agent versus agent speaker was not able to join us today, so thank you for staying with us. As far as the questions go, I know you're using the agent you gave a scenario about, like a ticket.
If the ticket gets resolved or change the. I could see that and go check that. That bring up other security possibilities that maybe the agent becomes vulnerable itself and it can't read that. Yeah. Regarding that. Yeah. So that's the thing right. The agent becomes part of your system. So it might be vulnerable to a range of things. Right.
Like you might be vulnerable to prompt injection. That's all things that you have to take into consideration when when putting those together. So definitely. Yeah. And I can show you man. Like I have my laptop here with me. I can show you some of the stuff that I did.
All right. Any other questions otherwise. Well we yes we do here. First of all, thank you for the presentation. And, I wanted to ask, have you looked into the some of the open source project, such as Kai? It's a, framework for agent. I have offensive and and defensive cybersecurity. Yeah. So I've looked into it. I'm still playing.
I start looking into it next last week, so I have played just a little bit. I do not have enough to form an opinion on it, but I've been playing with it. Ask me in like 2 or 3 weeks and I'll, I'll give you my take because like, right now it's too early for me to say anything. So yeah, I, I've no event.
I've been, I've been playing with it, but it's too early for me in the process to, you know, tell you anything about it. Thank you. I think we have time for one more. And I see over here first.
Two questions. First one papers. Where can we find a dump of links for all those papers? Second part, in terms of, you know, maybe this is a human in the loop question or or. Yeah, that's probably a good framing for it. So.
If you have a human doing a task for another human, that human is likely going to self-correct. If they start, you know, they start down the task and they do like whatever, a couple of the first parts of the task and they realize they're going in the wrong direction. They're probably going to self-correct and, and get aligned. AI does not do that.
And it's really fun to watch it spin, but, maybe you have some strategies for helping the AI get itself unstuck. Or maybe, I don't know, maybe it's literally just stop the AI and start from a different prompt context. That's a great question. In, GPT five kind of self corrects kind of. But it's like, I wouldn't it's not like a human.
Right. Like it doesn't stop in the middle of the thought and goes back and try it again and it finishes and then goes back to the start to double check. You can ask it to double check the work. That's what I typically do. I ask you to triple check. So if you go into thinking mode and you expand the thinking mode and you can see that it finishes the task and it goes all the way back to the top and try again and, and revise itself.
The problem is it's not always that it does a good job at revising their own work. Right. So you can think of it as like pure programing. And you have like two agents that are checking, checking them or checking their work. I've done that in that phishing research where I generate the phishing email and I send it over to the another model.
The other model is going to check the the content for intent and see if that if if the intent of the email is malicious in nature, or if it's something that you could potentially say that, well, yeah, this would be classified as phishing. And it goes back and say, hey, you know, re reedit or edit this email so it looks less fishy, right or wrong.
Again, to look at species and consider this. There's three things here that will make it less fishy. And check again right. So you can you can kind of create a checks and balances thing on your workflow. And think in the context of like pure programing. And that minimizes some of the quality issues that you have. But in terms of like double checking the work, it's, it's not great, but that's what that's what I was able to do.
But I like to have people to look into things first. So usually my thought process here is that at the end of the critical tasks that you have in your workflow, you have a human in checking the quality of the results and and teaching the model to be better at that in the future. So reduces the workload on the human at the end.
So what I observed is that the labs are really good at classifying and generating content, but really bad at making decisions right? And I don't consider selecting tools as making decisions because it's basically routing. So you're not exactly making it's not a complex decision that it's trying to make. It's just routing. Right. Like you have a list. You have a list of things that you can do with that list.
And, you know, it's like going to the supermarket to do grocery shopping, right? Like it's not a it's not a complex task. So it's good at that, but at making proper decisions. It's I couldn't guide a model that was good at doing that. So I always have a human at the end of the loop. But I can get I can get you the papers, like ping me on LinkedIn.
I'll send you all the all these papers are publicly available, so I'll send a link to all of them and you can download all of them. So thank you again. We are finished at this time. So we do have lunch, available from 12 to 1 over in Hall A3. And I am reminded to remind everybody to please take one more pass through the exhibit hall and finish your passport cards, drop those off with the at the registration desk to be entered to win one of our prizes.
During the closing ceremonies, the exhibit hall will be closing at 2:30 p.m. this year. So again, thank you very much, and we'll see you back here at 1:00 for our next presentation.
