Presenter:
Transcript:
All right. So this track we have air socks beyond the hype, what it really takes to build an AI enabled sock by Stephen Morrow. Your speaker today is the Chief Solutions Office chief Solutions officer at air MDR, leading innovation in autonomous MDR. He's a former leader at Devo, advancing AI driven security and large scale analytics. Stephen is an expert in automation with a focus on improving efficiency and security operations.
He has a proven track record in solution engineering and driving next generation security strategies. And without further ado, he will take it away. Perfect. Can everyone hear me? Mike, ring. Thanks, folks. All right, so the talk today is, one that is focused not on necessarily presenting a product to you. Yes. I work for a product company, and we we've done the product talk a lot where we go around and we basically say, hey, let's go ahead and talk about like, why you need my product.
But the reality is we get a lot of people who just ask, hey, what if I want to go build it myself? And so that's what this talk is actually focused on, is if you want to build your own autonomous SoC, how you do it and during it, like I'm going to go through a lot of the, the challenges, the pains, the things we went through as we built our company.
So a little bit about AMD eMDR. First off, we are a three year old company. Two years was spent in stealth. When we built when we set out to do this, our goal was effectively to make an autonomous SoC where humans were at the center. But I was on the exterior. So as we go through and we talk about it, I'll kind of explain how that weaves in.
But ultimately that's what we what we created. We wrapped that in an MDR service. They told me this was funky. So you're right, it is good in advance. What if you could? All right. So one of the questions is why? Why now? Like why build an autonomous SoC at all? The answers are pretty common. I mean, if you've been in security for any period of time, you've probably heard all these things.
Alert volume. Right? We get a lot of alerts. Way too many. And they're difficult to manage. Tool sprawl. I mean, most of us have a just a ton of tools. And with that tool sprawl, there comes a huge knowledge issue, right? Because you've got to learn each tool and each process, and you've got a million screens to kind of add together the last piece of skill gaps.
Right? Which ties into that, like if I've got to know two sims and I've got to know 30 other tools and I've got to, there's a huge skill gap there. So as we looked at the autonomous SoC, one of our questions were, how do we address these three? And in particular I want to and I'll probably mention it more than once.
I want to talk about the alert volume piece. We've kind of done ourselves a disservice. And security, as was mentioned, I was previously an executive at ACM. And if you came to me and said, hey, I have way too many alerts and they're overwhelming me, the only answer I could really tell you was, well, you only have so much budget.
So reduce the number of alerts you're looking at or finely tune the alerts. Well, in a way I'm creating holes, right? I mean, if the mission of security is to protect the enterprise and I keep saying, hey, let's scope down, I'm going to create holes. And so when we went, when we approach this, one of the foundational principles we wanted to approach it with is let's stop creating holes.
Let's open the pipe back up, let's look at more signals. But let's figure out how to do it within a budget and figure out how to do it with fewer, with little to no people. Impact. Can you advance? So the first thing that we looked at was what are the outcomes like what do we want to why don't we see, what do we want to achieve.
As we move forward and what is obvious MTI faster investigations. So we created a benchmark for ourselves, which is for 90% of every alert that comes in. We will fully investigate that within five minutes. Meaning fully correlated, fully enriched, all you know, we know exactly what what what everything we could at that moment in time. And we have reached a predisposition to outcome of it's malicious, not malicious, whatever it might be.
And we want to do that within five minutes. So that was a goal we set for ourselves. And it's a goal we've actually hit at this point. There are outliers. That's why it's 90%. And when it comes to AI, for some reason, everyone's like, it's got to be 100% no, your socks, not 100%. It's got to be at least as good, if not better, right.
And so that's the target we aim for from a quality perspective. We wanted consistency. If you look at socks, you look at SoC analyst teams I've worked with at this point, probably thousands of them. They all do different approaches. And even between the SoC analyst, the level of consistency is it's what it is, right? I mean, you can only look at the same alert, believe me, all day long, and document it so many times before you will inevitably not do something.
So the consistency is a problem. The last one is reduce burnout, and that one is probably one of the most important. If we're going to increase the scope of alerts and therefore cover more of the attack surface, if we are if we just have humans do it, we're just going to increase burnout, right? So the question has to become, how do we take the mundane, the things that we do repeatedly as SoC analyst and automate that while leaving the human intellect and the other pieces intact?
And often at this point, I think it's a good, good kind of a good place to get this thought out of the way, because this is always where people start to think this. Am I advocating for replacing humans and security? No, I do not believe I can replace humans in security. And the reason I say that is because when you look at security, it's kind of unique as an industry.
If you look at like tax accounting and you bring automation into tax accounting, the rules are defined, right. You have books, you have operations by which tax accountants work. And even if those change, you simply update the lexicon. But it's a defined set of rules in Lmms. And I are very good at operating within a defined context. Security is not you've got a human on the other side, and they are seeking and in intending to subvert the rules, which means the rules themselves change.
And the only way to counter those rules is to make sure you have the human at the middle. And so that's why, at least in my belief, I don't believe we'll ever fully replace the human, because we're always seeking to use humans to supplant humans in the in that security race, if you will. Can you go to the next slide?
So what an AI powered SoC means to us? I mean, I've obviously talked about human in the loop that that is a huge piece. The question becomes at what point in the loop. And for us, the point of the loop is the end of what I like to call the monotony. Right? The moment that an alert comes in and it's been fully triaged and all that's occurred at that point, if it's not automatically closed, which you only want to do if you have a high level of confidence, then you put a human.
But at that point, instead of teaching people as SoC analysts how to document cases, how to write it up properly, how to go find an IP and enrich it, we're really just more focused on user security skills. Read what occurred and decide if you agree with it. And more importantly, since we are an MDR service in our case and we have a SoC around it, we teach our analyst to actually do two things.
Solve the case as you would a security analyst, and then teach the AI. Ask yourself the question at the end how do I never work this case again? And once you ask that question, then you update the AI so that you don't have to work it again and you will never get 100%, but you might get 95 and that's pretty good.
The mundane I mean, I've said it like a hundred million times. One of the ways we got out of the mundane, though, was we went to natural language. So we have 200 plus integrations that we put behind the scenes, and we built a natural language saw around it. We use the natural language saw to build playbooks based on skills.
So they are it's not interpreting the language each time it's doing it once solidifying. Yes. You agree with that interpretation and then that becomes a playbook effectively. And that's for both investigation and automation. And so that does a few things. One, it eliminates the getting wonky and doing something weird that you didn't expect. Two, it allows you to very rapidly innovate, and it allows you to get out of the mundane by plugging in things you wouldn't plug in otherwise.
So take HR data as an example, right? Like we can enrich with people's geolocation if we need to. And maybe that's in an HR database somewhere and maybe it's all over the place because maybe we've done some M&A. We've we've acquired some companies and we have lots of different products. But with this, because of the natural language playbooks and integrations, I don't have to do that by hand.
So if you're considering building this yourself, you have to start considering how do I integrate with the sources? How do I eliminate the mundane? Next slide please. You have basically two options today for models. If you want to go down the route of building it yourself. First option public OEMs. These things are great for certain reasons, right? First off they're there.
They're available. You don't have to set much up and plug right in. You can get commercial contracts. They're they have strong capabilities are evolving element in ever increasing rate. There's some cons most of them are hosted in the US and only in the US. So you have some data residency concerns, things like that. If they decide to change the model and or the parameters and or the API, you're a slave to that.
You have to change with them. You have to follow their emotions. And so as a result that has a it has a cost. The last piece is the unknown future cost. And this is a big one. I assume people here remember when the cloud was supposed to be cheaper? Is it pretty sure it isn't. That will happen here to just give it some time, like it's inevitable.
So that's another factor you have to account on. The second option, if you guys could go to the next slide, is hosting your own. Right. So you can do private open source. Well there's some good things there. Get data it controls and sovereignty customization. You could air gap but you otherwise can't do but you do have to maintain it.
You're vulnerable to attacks on those boxes. Like, because you're you're now the security team for those boxes. You have to have an MLOps team to maintain them and adjust the models, fix things, change the model when that gets upgraded. So you end up owning the whole process. You have to pay for the compute. You have to understand the compute.
So there's a cost to that as well. If you look at the two and you ask the question which is better, I'll tell you the question. We arrived at both. So in our case we have several. I'm just going to refer to them as secret sauce hosted models that we use. And we have several public models which I'll disclose publicly.
OpenAI anthropic. And we use those under commercial contracts, and we mix them depending on what they're good at. And we also mix them in depending on data residency and other such rules. Right. So the engine we wrapped around it, facilitated that. Can you guys go to the next slide. So then you get to the question of how do you make it usable and safe?
I will tell you, SoC analysts want to do one thing and it's solve security problems. What they don't want to do is code. They don't want to get deep in the weeds on things that are ancillary. They don't want to have to go learn yet another console. They want to solve those problems. When we first in stealth started showing the product, we didn't have a Google.
It was API based, right? Which makes sense. That's what you do as a startup, right? You build something and it's like, hey, here's this wonky thing, what do you think? And they loved it, except for the fact that they didn't want to use it like they wanted to play with it. Right? They want the tool with it. Us.
Neat. Now I'm done, right? Because it was hard. It wasn't fun. It wasn't easy. It was just painful. So you got to start thinking, how do you minimize cognitive load? How do you make this? How do you how do you avoid automation bias? How do you wrap this thing in something. And so in our case, of course we built it gooey.
That's what you do. So we're SAS hosted. We effectively built a simple plane for them. We gave them the ability ability to drop down. And we made all the controls. And they're very explicit for the humans. So they know where they can click what each thing is. And effectively they control the entire process through natural language. You don't have to learn a new tool.
And interestingly, you don't even have to learn a new language if you speak Spanish or Greek or Italian or whatever it is, Elohim doesn't care. They will interpret it and that'll get translated to skills. So it provides a great degree of adaptability, and it lets people focus as an analyst on purely their ability to solve security problems. Instead of focusing on the how do I get there?
You have to do that. If you don't do something like that, people will not adopt your platform. The next piece is integrating feeds. You can't just ask them, okay. And now as a manual step, please go out to your system and do the thing like that's not the point. Right. So you've got to start to integrate APIs.
In our case we've integrated 200 plus APIs. I do want to put a note in here. If you decide to go down this path or an MCP and A to a, those are both, model context protocol agent to agent. Those are things that the industry is adopting. They are relatively new, in my personal opinion. They're not quite ready commercialization wise.
They're definitely ready to play with. They're not quite ready for how you commercialize them, mostly because either you have to host them or they host them, and then who's going to update them. And a lot of that in the industry hasn't really been worked out. But if you're going to go down this path, you have to consider those and you should be thinking about those in advance, because MCP does provide an enormous amount of power, in the fact that you can the models basically talk to each other and then decide the API calls.
So you obviously need a lot of stuff. And if the if the vendor, as an example is constantly maintaining it, then it's a good thing. Actually, I'll give a shout out to, Lena Charlie host and MXGp server and it's fantastic. So there is some there are some good options. There. Go ahead and go to the next slide for me.
Thanks. So the correlation piece becomes a really sticky wicket. And every time I bring this up at first people's head spin. Most of you probably group alerts and categorize or loops or alerts in groups. And you do it because it makes sense, right? You have an entity or a user or something. And I'm going to group these events together and say they're related.
Yeah, it no longer makes sense in this model. Instead, what starts to make sense is grouping cases. And so you're thinking, well, why would I group a case? That means I've done all this investigation and this benign thing. Exactly. You have. And at that moment, you've captured all of the enrichment. At that time, you've captured all of the data at that time, and you've put it into a case, and you did it with no additional human manpower.
So it cost you nothing. So then you have to start asking yourself the question, should you correlate on cases or correlate on alerts and answer correlating cases? Because then you have fully and fully enriched alert stories that you can connect, which are way more powerful than going back six months later when you realize something malicious has occurred and trying to figure out what it was at that time.
So it's it's hugely powerful and it's there's a big paradigm shift. And it wouldn't work if it was people, because then you'd be investigating a bunch of the nine cases. Facts and investigation notes are very important. In our case, we're operating this. We actually sell it as an MDR. We also sell it as a service to MSPs and larger companies that may have advanced SoCs.
You have to be able to when you're especially when you're selling a service or you're doing M&A or things like that, to have a concept of facts and what we call investigation notes. So facts for us are holistic. They effectively are things that influence the lens behavior. So I'll give you a real working world example. I have a client that has three separate HR systems.
I love picking the HR systems and two of them have the wrong geography geography data. And they told us that they're like this. We don't keep those up to date. We're really lazy, just this one. So there's a simple human fact in there, just, you know, English. This is, hey, when you're investing in a case and you're going to go fetch this stuff based on the integrations, you have to ignore those two and always prefer this one.
But in the event that you know this one is not populated, do this. And it's literally worded just like that with that one line, it changes the EMS output, it changes the other one's consideration of the facts, and it changes the way that it that it adapts. And we do that at like a tenant level. So we have a concept of like doing that at a higher level too.
So if I do M&A or I acquire a company, I can come in and adjust that. So as you're going through that, start thinking about how you do that. Also think about a concept of investigation notes, which are the same kind of the similar to facts or human language, but they happen at the technology level. Right. For a CrowdStrike alert of this type, behave this way for a alert of this type, behave that way.
Because as you get into phishing alerts versus, you know, some other alert there, they're all different, right? Identity alert versus phishing or different. You need the different facts. So you do have to consider that as well. You definitely want to make sure whenever your AI is making edits to your case or you're making it, it's your case or anybody that you're tracking the artifacts.
One of the key purposes of security, beyond protecting the enterprise, is providing legal evidence in the event of an incident. And if you're going to introduce AI, you track it like a person. So keep that in mind. Working alerts holistically was one of the big mistakes we made, when we first started. When we first started, we started thinking.
We started with the what seems like the obvious solution here. I'll just give the AI an alert and make it work with this. Let me know what happened. What you will quickly find is it will do the state right. It will start to solve the alert in different ways. It'll never give you the right answer. It's just kind of all over the place.
And so what we learned from that is you don't over trust in it. And one of the things you do is you break it into small pieces. So we've actually with those natural language playbooks I mentioned, broken into little pieces that allows the engine to investigate in a step by step manner. Right. Go do enrichments, go do this, go do that.
And I kind of you learns a lot like children. Like if you ask your kid to go make a peanut butter, peanut butter and jelly sandwich and you're like, hey, get the bread out of the fridge. And then it sets it on the floor and you're like, what are you doing? And then they grab the peanut butter at their hand and you're like, dude, use a knife.
Like, you know, it's one of those things that you have to be very explicit in your instruction. So breaking it into small pieces, if you go down the path, very important. Like you will want to kind of kind of instruct it in that way. Could you please, when you start to customize you, you know, we've talked a little bit about investigation notes, but you have to customize it for you.
In our case, we built a platform this customizable to a lot of people. Right. Because we're running a business. Right. This is an MDR. So I have to be very flexible. But in your case, you have to you may not need 200 integrations. You might need seven. So build that concept as well as look at how you need to holistically organize organize.
And when you think about that, think about your organizational structure as a company, because that's probably how you need to organize. Right. Security often follows those same organizational paths, like who's allowed access, things like that. So if you organize your system in that same way, it will help. I've already beat M&A to death. Like how you can do that.
So I'm going to skip that. Can you guys advance? Workflow. There's more to security than just investigation. We've talked a lot about that. And the reason we talked a lot about it is it's very easy to automate. And in our case, like if an alert comes in, we are genetically meaning we basically auto create a playbook. I don't do that for remediation because that would be dangerous.
So instead for remediation I create the playbook from the natural language instructions. You improve the playbook and then it becomes a thing. Right? So it's not it's controlled, it's known. It's not going to go off and do something weird. You need to think about that though. How do you if you're going to automate investigations, why wouldn't you use the same tool to automate remediations, which is the same conclusion we arrived at?
As far as handoffs go, you need to think about case management. You could purchase another tool. It's an option, you know, ServiceNow or something like that. You can also build your own. You have some options. But do consider it. Don't leave it. Don't get it all the way to the end and then be like, oh, now I need a case management system.
You need to think about that in advance. Make sure you have that in place. Make sure it plugs into the AI models you're picking or the platform you decided to build. Chain of custody. In our case, we broke all of the chain of custody out into a separate system away from the AI, because I don't want any chance of it making changes to that.
It's very important for me to know who changed the case, who altered evidence, including if the AI alters evidence, I want to know why it altered evidence, what it's reasoning was, things like that. So we actually broke that out. I would suggest you do the same. Because if you're ever called in the court, the very first thing any good prosecutor's going to do or any anyone on the opposing side is going to say, hey, how do you trust that I.
Right like and it's a fair question and you need that documentation. It needs to be immutable. It needs to be preserved. Next slide please. So we've kind of gone through like you can pick a model and kind of how to build the system. But now you've got to keep it maintained. You've got to measure accuracy. Just like every tool there's a maintenance cost to doing this.
This is not free completely. Right. Like it's not it's not a free pass. Right. You still got to maintain it and look at it. So some of the things are you know you've got to judge in case output accuracy. Right. Like I said people love to shoot for 100% here. I think that's pretty foolish. Your socks not doing 100%.
I mean, we're humans. We make mistakes. Like all of us don't care how good you are. So shoot for a reasonable level of accuracy that you can hit. You need to make sure those enrichments are consistent. They're happening in the same way. You're getting the same data back all the time, like it's it's functioning correctly. And then of course, the usual, the usual KPIs.
You want to keep it, keep an eye on those empty into all those fun stuff. From an ongoing verification perspective that becomes how do you do it? Right? So obviously you got to sample the cases that are being produced. But I highly recommend red team drills because one of the things you're actually doing is teaching the AI how to be a little better.
Right? You're going in and you're you're you're providing that. You know, when I started this talk and I was mentioning how humans won't be replaced, you're actually providing some of that data to the yellow. And so it gets better as you do more and more red team. So red team drills. Great correcting for limb drift. Pretty important.
Limbs will drift. They will kind of, decide that today. This is that. And move over there. And so as you examine these cases and as you do these red team drills, you're going to pull that out of the model. When you pull those things out, you're going to have to start to, add counter facts back to kind of it's just like nudging it back the other way.
And that's that's how we do it. So we had facts and say, yeah, I know you thought this, but don't think that any more. Think this instead. And you've got to do that every now and again and just constantly force it back into the place it needs to be, because it will go in odd directions. Next slide please.
So planning for the change in the real cost. Well, you got to think about your hosting and all of them cost. Those are going to to have have an impact. There's definitely an API cost in terms of the upkeep, right. If you're a larger enterprise and you've got 190 tools, you're going to have an API cost.
If you're a smaller, you know, smaller outfit, the cost will be less, but you still should still consider it as well as some vendors call it charge for API costs. There's API limits. There's all sorts of things to consider as you go down that path. Cost of the up I evolution speed is a is another big one that I'll, I'll dive into.
This thing changes like daily like they're, we're shifting at a, at a rate that is incredible in this in this space right now. And so like from the day that, I actually have only been with MDR since February, that's when we started or when I started with them. But in that time we've effectively rewrote the product. I mean, at least twice.
So the change of the AI side is just pushing, pushing radical change very quickly. And as a result, you may be able to keep up with that, which means you need a large engineering staff. You need you need to be planning ahead. You need to be making those moves. So there are definitely some cost to that, that you need to consider.
You also need different people. I did have, a socket manager I was talking about. He was like, hey, I'm going to build this myself and decided to use his sock to build it. And I'm like, yeah, that's probably going to work like you do need people to have him ops, you know, kind of capabilities. You probably have the people that can do like detection, engineering, things like that.
You probably don't have data platform in UX sitting in a sock. I mean, like, if you do, cool. But that's not the norm. So you do need to actually think about I need an actual engineering team and a SoC, if I'm going to go down this path now, obviously there's an alternative. You can buy it from somebody, such as myself or other vendors who sell it.
But if you decide to build it yourself, you have to consider that, and then you've got to budget for the eval pipelines, data set curation. It's not a one and done like you are going to be maintaining this thing. In the case of an MDR such as ourselves, like we maintain it, right? Like I'm a SaaS platform from a platform perspective, and I have a bunch of humans who act as a SoC around it.
And if I'm selling in a platform motion, then it's a it's a tool, right, that I maintain, which is great. And if it's a, you know, more of an MDR motion, then it's like, hey, I'm selling you the SoC and the tool, right? And they're kind of wrapped into one next slide, please.
Can you guys have the next slide for me? All right. Awesome. Build versus buy proof points. Obviously if you're going to get money to build, someone's going to pay you to do it. Someone's got to provide the cash. So how do you kind of work that angle and get the money to do it? Well, one option is you go buy from someone else, you get MDR off, you know, you get 24 by seven operations for like an MDR, you get accelerators.
I can kind of do things in bulk, so I might get a level of discount. You don't get that kind of thing. Another approach if you do it yourself, is hybrid. So maybe you own part of the system and maybe you plug in to plug in halfway. Right. So we have some approaches like that where one of the things we encourage is like, I don't care what tool stack is, I've got 200 plus integrations you want to use CrowdStrike or Sentinel one.
Defender I don't care. You want to use I, my MDR supports my my actual SoC supports ten. Seems like it's fine. Like I'll make it 11 tomorrow. I don't care because it's all wrapped behind the engine. So that is another piece of like, how do you own your tools? Which tools do you want to own? Which tools do you want that don't, you should be hitting 90% if you can't, if you can't convince whoever's giving you the money, likely an executive, that you're going to hit 90% or better, they're probably going to give you the money.
So you need to make that a target. And you need broad enterprise adoption. And I think that is a huge kicker right there. There's a lot of people that pull back on this, and I see a lot of resistance where it's like, hey, this is going to cost me my job. It's not. And you need you need to be able to effectively paint that picture.
If you can't convince them and show them that this isn't there to hurt, it's there to help. Especially the level one. Guys, we've done a horrible job of converting level one analyst, a level three analyst in the industry because we spent more time teaching them how to, like, write a case up and then slapping their hand when they write it up wrong than actually doing security work.
And so the conversion rate is really not great. So if we could turn around and you can get broad enterprise adoption and people start to use the tool, you end up accelerating the level one guys and they become, you basically just end up with all the little threes, which is fantastic. And I know everyone has different levels, but three good one entry code that, so what does good look like?
Start with your workflows and KPIs. Pick a model, make that window go away. Thanks. Pick a model strategy and set governance. Invest beyond the Lem. Don't forget to invest in the people treat. I like critical infrastructure backup Dr.. Fell overs multiple models right I don't just I don't have multiple models just for the reason of they're good at things.
I also have them for fell over. Make sure you keep human in the loop. Governance that data. Evaluate the skills, measure the KPIs. Next slide please. Okay. So in a second I'm going to take questions. Before I do I want to talk to you about the toolkit that's in the back of the stack. So we built we put a huge toolkit in the back of the deck.
You're welcome to a copy. You can get it from our booth. There's a gentleman named Andy somewhere. Where are you at, Andy? There he is, right there. He can take your email and such after the questions. We're happy to have a chat. So those are all options real quick before Andy chimes in, because he always does.I want to go through these just kind of flash through these next slides, guys, just like at a reasonable clip, just so you guys can kind of see what the toolkit looks like. Actually, I can probably walk back here and try it, see if it'll do it. The range on this just sucks how they're doing it. That's perfect.
So this is basically what you guys can take back. And it'll give you kind of an idea of what you should be measuring, what you should be picking. Flip there a few more guys. If you could. Yeah. There's a phase checklist here. You can see various metrics in there. Those are those are all there for you.
I'm not going to go through these in detail next one. And then obviously like how do I validate it. Like so we walk you through step by step effectively. Like here's the things you should look at. Here's the things we looked at. And then consider those. And if you can answer these questions and effectively you can go build it.
As long as you have the budget and whatnot to do it. So from there, I'm gonna let Andy chime in and then I'll take questions because he loves chiming in. There's a mic, Andy, if you need it. Okay. No thank you for, adequate. Awesome. Thanks. Adequate is a great rating system. You like I. Want to make.
My. His forehead scares a lot of people away. It's shocking. So many different. You may not.
Investigate.
But yeah. You in the forehead. All right. We make fun of us for. Had a lot of fun. There's a mic there. If anyone wants to ask any questions. Happy to take them. Yeah, go for it. You can use the mic. You can speak, and I'll repeat it. Whichever works. I'm not that loud. And you do increase the size of my forehead.
Anyhow, you've mentioned some environments. This is not a great fit for. What are those environments? Generally, if you don't like, I have customers that'll come to me and be like, hey, I have no no tooling stack at all. Like, I don't have any need or I don't have a SIM, and I'm just getting started. If you're in that place, don't worry about automation, man.
Get your stack in place. That's one. Two. At this time, if you have incredibly strict data residency requirements. So like I don't store data in the LMS, obviously. Like I just pass it through them for certain industries like defense, right? I can't even do that. If it's out, if it's not done in the US. So things like anthropic open AI, right, right now, like you're kind of locked into that.
So that's another area where it may not be a good fit. Those are probably the two biggest that I see. And like I'm particularly thinking when I like the U.S government is somewhat okay with passing data through NLM in the US. Like like the Australian government, for instance, is not like I can tell you from experience, they are not okay with that.
So that would be an answer okay. Underlying infrastructure and data residency almost entirely. Those are those are the main reasons or the fact that you don't have the tools. Thank you. Sure. So so output, open source and a little all the stuff, compared to, public. Okay. Which I think is, but so sort that.
No, I'm not good at like, think that is how we can make those, open source a little better for this. Okay. So there's there's a few things to consider here. So the question was open source LMS versus public LMS, which is better? And if I get it wrong, can tell me the second half was how do we improve the open source side of it?
How do we make those elements better? So I wish I could give you a really straight answer to open versus public, but I can't. Even if you look across the public models between like, the newer models that ChatGPT has versus the models anthropic has, they change too frequently, so you'll end up experimenting. The other thing that you'll notice is when it comes to the investigation, some are really good at summarization as an example.
And you'll be like, oh, I'm gonna use this for summarization. And some are better for logical thought. So you're going to end up picking a mix of models. You could hypothetically go completely open source. The reason we didn't do that is cost effectiveness was probably one of the big reasons. It is actually more cost effective to introduce a public LLM for some tasks than it is an open source, at least today.
But the way that my model set up, I'm able to shift away if I have to. If the cost goes up for open source models, there is one concern I want to kind of call out right away, and that's model poisoning. If you're going to go down the open source model route, be very knowledgeable about the model. You're choosing.
Because there are tons of awesome repository repositories like Hugging Face, where people love to go in and poison models. So make sure you know which model you've got, make sure it's the right model, make sure it's it's not sending things or doing things in ways you don't want. But as far as improving open source models to do this task, I think it's more I mean, unless you're contributing to that model directly, there's not a lot of ways that I see to really quickly improve it.
We're using a lot of general purpose models and then using the platform to kind of constrain, instead of trying to build a specific model for this specific task. Does that make sense? Yes, I am sure that. So anybody who's in public or as well makes public and private. But sometimes for some companies, when, all of my long years of waiting to open the doors for a public, the more susceptible to digital, the, but the from with the public, the they didn't lose all that.
So then the value is, you know. Yep. Right. So so for you, it's actually how you handle the situation so that there's going through the public cloud. So for public islands, we have commercial contracts with OpenAI and anthropic. And that's how we're preventing that using that model data for training or anything like that. If you're going to build it yourself and you're not going to get those contracts in place, then you need to go do open source to your point.
And that is one of the advantages of open source. As I mentioned, you can air grab it. You're not going to air gap with OpenAI and anthropic like that's just contrary to the idea. Right. So it does depend on what you're trying to do. Which one you choose. But you're right. You don't want the data going up there without a commercial contract.
I guess as a warning, don't put your data up there without a commercial contract. Like you shouldn't be doing that. You have to have a commercial contract if you're going to be safe about it. Other questions? Anymore? Andy, you got a question? No. All right. Awesome. Then I appreciate it. Hope you guys enjoyed the talk.
