Freddy Murre: 0:00

Basically, everyone just do whatever they feel like and then call it intelligence.

Rachael Tyrell: 0:05

Hello and welcome to Episode 18, Season 1, of your Cyber Threat Intelligence Podcast. Whether you're a seasoned CTI expert, a cybersecurity professional or simply curious about the digital battlefield, our expert guests and hosts will break down complex topics into actionable insights. On this episode of Season 1, our host, Pedro Kertzman, will chat with Freddie Murr, who is an intelligence professional with more than 13 years of practical intelligence experience from both military and private sector. He is a prolific speaker on topics like intelligence methodology and tradecraft, CTI, how the CTI community can benefit from using intelligence tradecraft, and how AI may change how we do intelligence and CTI. Freddie holds a BA in marketing, one MA in Counterterrorism and one MA in Intelligence. He is currently doing his PhD, where he is researching the intersection of intelligence tradecraft, CTI and AI. Over to you, Pedro.

Pedro Kertzman: 0:56

Freddy, thank you so much for coming to the show.

Freddy Murre: 0:59

It's great to have you here, yeah finally, we've been trying for a couple of months now to sync our calendars, so it's great to be here.

Pedro Kertzman: 1:07

That's right. That's right. Would you mind, you know, doing a little introduction to the audience how your CTI journey started? Sure.

Freddy Murre: 1:14

So my name is Freddy, obviously Freddy Murray. Anyone who knows me on LinkedIn will probably see me speaking a lot about intelligence and stuff, but sort of um. So I work at an organization called nordic financial search. It's a non-profit. That's sort of my day job. That's 50 of my time and the other 50 I work at the university and doing a phd and the research is actually how can we apply intelligence methodology, intelligence tradecraft, into cyber threat intelligence? So a lot of people sort of forget that the CTI cyber threat intelligence actually has a huge intelligence component to it. So that's what I'm researching trying to figure out how to do it. And also AI how will AI change how we do intelligence today? Ie, then, how will intelligence and AI together help us do intelligence better in CTI? So that's sort of the I wouldn't call it a threesome In Norwegian it makes more sense, but it's sort of the three parts of the research.

Pedro Kertzman: 2:17

Fair enough, and so you were talking about the super important aspect intelligence within CTI and any experience around how international institutions usually leverage CTI or European institution leverage CTI Any insights around that?

Freddy Murre: 2:37

Yeah, so a lot of the things we do is collaborating with various national but also international organizations. So since I work in a nonprofit, then we actually cover five countries in Northern Europe and through that, we also are members in different ISACs like FSISAC, fviasac, we are collaborating with law enforcement in Europol there's a lot of things going on but also with various other sector and other sector certs and other NCCs across Northern Europe. So we connect with and collaborate with a lot of different organizations, both on the government side, but also the private sector and when we work on CTI and perhaps I should go back a little bit and talk a little bit more on what we do and what I do, so we can connect the two. So I spent I don't know 17 years or so in the armed forces and almost eight or nine of those with intelligence, and when I got out of the military intelligence side, I started working at NSRC actually almost oh, what is that? In 2018, six, seven years ago and everyone was sort of I usually say they were talking the talk, but they weren't walking the walk. They were talking about intelligence, about requirements, about stakeholder, about intelligence requirements, et cetera. You know we got to do this, we have to do a collection, we have to have planning, and I was like, hey, this is. You know, I was pleasantly surprised about the lingo, about the talk, but the thing is that that was just it. It was the lingo. They were not using the intelligence methodology in a good way. Those few that were doing it were writing blog posts and telling others how to do it, and I was like, no, that's not how you do the intelligence side.

Freddy Murre: 4:33

Basically, what I see is a lot of the CTI teams, but also vendors, are doing cyber threat information, not cyber threat intelligence. You know so. So, um, although a lot of these reports are pushing data, they're pushing information the sort of the, the metadata, if you will, about data. So that, combined to me at least, is information. So when you have a list of ip addresses with no other information about the dates or last seen, the IP addresses connected to certain campaigns or certain time periods, and then they switch to this IP, why did they do that? What is the consequence of doing this? What are your assessments based on what you're seeing? All of those things are missing in the vendor space and that is sort of in air quotes. We say that it's inherited by the CTI teams. Right, this is sort of the way to do things. So when we look at the different teams, when they work, a lot of them are using the tools that the vendors are giving them.

Freddy Murre: 5:43

But the problems with the tools is, you get the panel and this is a list of all the indicators, but what are the indicators? That is relevant to me, with my technology, with my issues, with my requirements. There aren't any, or very, very few, vendors who are actually thinking about you as a paying customer, adapting their air quotes again, intelligence to you and your needs. Too often we see that you get this package, as I said a moment ago. Here's a list of indicators. These are bad go hunt. It has a value in itself, but why am I spending time on this if I don't have the technology? Why am I getting this alert if I don't have this technology? Or why am I getting this alert if I don't have on-premise? I have cloud. This is an on-prem vulnerability. It's irrelevant to me. But then again you have managers and CISOs and everyone reading the news. They're screaming in your ear. You have to have something to communicate to them, of course, but that's the huge gap usually we see is vendors coming with a lot of information and claiming this is malicious to everyone, where actually it's not malicious to you because you don't have the technology, or you are secure in a different way because of the technology stack and the defensive posture you've set up.

Freddy Murre: 7:16

So that's one side of it, the vendor space. The other space is, to me at least, is the teams, or other teams. When I say teams, I say that a little playfully, because CTI teams usually is the one guy or one person who has been given the title, given the role of being CTI. Hey, you worked here now for three years. Seems like you love this. Hey, what do you think about being the CTI guy? And you go. No sure you know that sounds fun, but that person has no background in cyber threat intelligence. That person might be really good at instant response or really good at detection engineering, but maybe not as good in cyber threat intelligence. So what usually happens is that those people will fall back to whatever they know, which is the technical part. It is how can I figure out the IP address? On this thing I read about in the blog post of this vendor. They're really focusing on that part rather than saying focusing on that part rather than saying these IOCs are irrelevant to us, because and then focus on what really matters figuring out which threat actors are actually trying to breach you, which threat actors are actually posing a threat to you.

Freddy Murre: 8:39

So if a vendor comes out with a report about activities in Southeast Asia and you have no activities there, you have no satellite offices there, you have no vendors there, why should you spend time on it? And I'm using this analogy because we've seen it so many times. Also, when Ukraine and when Russia invaded Ukraine, there was a lot of reporting going on about tanks, about soldiers, about you know when will someone do this or do that in the military space. And here you are, as a CTI person, informing your CISO about how many tanks have been moved and you know what is the operational status of this battalion. And I'm like, holy moly, this is so strange, right? This is not what cyber threat intelligence people should be working on.

Freddy Murre: 9:33

There's a counter argument to that, of course, when the boss comes in and say, hey, I'm a little worried about this conflict. What can you tell me about it, about this conflict? What can you tell me about it? Sometimes, you should, of course, provide information and provide support. Intelligence is about decision support, reducing uncertainty, providing the knowledge but also the assessment of stuff that is uncertain. That's sort of our job. So when they come in and ask about stuff they care about, yes, we should listen to and adapt to what they need, within reason. Usually that's what I say. So if it's too far away from what we are supposed to do, we should not give a pushback but at least sort of say, hey, we can tell you about the cyber elements. But at least sort of say, hey, we can tell you about the cyber elements, we can tell you about the campaigns and activities and how that has affected, you know or could affect us and our security posture and our sort of detection capabilities. That makes complete sense. But then starting to talk about you know, battalions and training level and how many tanks and planes and aircraft in general, it's sort of getting into the sphere of. That's something that perhaps someone else should be doing.

Freddy Murre: 10:54

And I saw a post the other day. A guy was using the analogy of the boss comes in and asks a question and he adds you know, maybe this is a little outside of what you do. And then the reply from the CTI guy was no, listen, you have a Ferrari and if you want to take it out for a spin, you take it out for a spin. You know you can drive it anywhere you want. My counter argument is well, yes, you can drive a Ferrari on the Autobahn and have fun, or you can drive it on a small dirt road with holes everywhere. That's going to be a poor journey for everyone. The car is going to be destroyed, right. So it is fit for purpose for certain elements, for certain roads and for certain sort of speeds and atmosphere, but not for everything, right? So, yes, you can drive around and do what you want with it, but it's going to be good for one thing and bad for another.

Freddy Murre: 11:50

So I think a lot of the CTI teams, they should focus on what they're doing. They should be able to support the SOC, the insert responders that's their core function in most of the cases we see and hear about but also verify with your key stakeholders what else is it that they want you to do? If they want you to support on risk assessments not the risk assessment per se, but information into their risk assessment Well, that's something different than giving them all the IOCs. They don't care about the IOCs, they care about the. So what, when you have seen these things, these reports from these vendors, how can that affect us if our detection is not good enough or if our defensive posture has these certain holes of vulnerabilities? That's where you really have to be able to communicate what the vulnerabilities means for the business, what the threat actor and their motivation and their skill levels, what that means for us and our detection and defensive posture. And if that's challenged, how could that, if they were to be able to bypass? Let's say, a ransom incident happens or someone steals all our data and it's a lot of intellectual property, what would that look like and how can we then detect and respond on it? How long will it take and what will it cost, et cetera, et cetera. Right, so now we're getting into more of what the business cares about rather than what we care about, more of what the business cares about rather than what we care about.

Freddy Murre: 13:25

And I guess too often we see CTI people writing blogs and reports for other CTI people. So for one level it works right. I see something that is really relevant to everyone who does CTI, or at least some people who are in the same have the same issues that I have. So it makes sense for me to write to them. But, again, those are not all the readers, not all the consumers of intelligence.

Freddy Murre: 13:56

Usually we say there's three levels. We have clients, customers and consumers. Consumers is anyone who might be interested, who will read and consume your reporting? Customers are people who come in every once in a while and sort of buy in air quotes intelligence from you. You don't have an ongoing agreement with them. They come in every once in a while and want something from you.

Freddy Murre: 14:22

Clients, on the other hand, are the ones that you are delivering to continuously and it's the reason for you existing. So if you work with CTI, those are your key clients and the challenge here is that for one type of topic or question, you might have a set of key clients and some customers that you have delivered to, but for a different topic, that changes. And understanding and being able to engage with the different stakeholders at different levels and understanding what their needs are that's a huge, huge thing where a lot of the teams I see misunderstand and sort of come running with again. What I said initially when I started this rant was that you know they fall back to the level of their skills rather than you know, excelling and doing something completely different, which never happens. You know, to be fair, that was a long rant. Hopefully.

Pedro Kertzman: 15:23

No, that was awesome, I appreciate it. You touched a long rant, hopefully. No, that was awesome, I appreciate it. You touched on a very good point.

Pedro Kertzman: 15:27

I would say the industry in general is set in a way that the vendors do a one directional IOC information to customers you mentioned.

Pedro Kertzman: 15:41

They don't understand the reality of the customers, they don't know if it makes sense for their environments and their operations, and all that, I think, because technically, everything is sitting on a taxi's server that is created to shoot information one way, so they feed that to the customers. Here you go, do something with it, and then the brainstorming part is what, if kind of this is the core problem, and instead we should be using more like a you name it like a bi-directional API. So we first yeah, we do send some information, or vendors, or the vendors send information to the customers, then the customers will react to it hey, this makes sense, this doesn't. And then the vendors, based on that information that they will receive back, they can fine tune the initial, and then we would be closer-ish a little bit to actual intelligence not there yet, because we still need somebody with a lot of proper context on the customer side, but at least it's not just dumping information on on on somebody. But what do you think?

Freddy Murre: 17:03

about that. The interesting side there is that in the cloud space I've seen some of the things that some vendors are doing and there's one I'm not going to name names but they have a really interesting threat intelligence module where every night they scan the network and scan sort of the system it's connected to and just updates its list and inventory of systems and versions and you know whatever the technology they find, and then they merge it with what are all the vulnerabilities that is existing out there that is known, and then actually warning the client when they say, hey, this vulnerability, you have this technology and you need to do something because of the configuration we've seen, you're actually vulnerable here. That's really, really valuable and that's what the cyber threat Intel vendors should be doing of the technology stack of their customers so they can actually support and actually provide value rather than just pushing through the panel. You know these are the seven reports we produced last week. You know, great, but they had no benefit for me whatsoever because we didn't have the technology stack or, you know, they didn't affect us in any way, necessarily in the way that at least we should care about. So I see that, um, that there's some growing up to do, I guess, in in that space.

Freddy Murre: 18:35

But then it creates another problem, right? Do a bank or any organization for Any organization for that matter want to give that type of access to an external vendor? A lot of the time, no, but in the cloud space they say yes. So there is some interesting discussions there, right? So if in the cloud space, cloud customers say, yes, you can scan everything in our cloud system so you can have that information and provide us value, if they say yes to that, but you can't scan our physical network in our own building, why is that? What is stopping them from saying yes there? Because there's shadow IT, know there's poor access management. You know certain uh, people have way too much access. People have quit and still accounts are valid or active. There's so many bad things going on that having a vendor coming in and sort of telling you to fix all these things because you have forgotten, or the admin who had that control quit two months ago or two years ago and the only person who knew was that person, right, so there's so many things that could be fixed here.

Freddy Murre: 19:54

I'm not saying that is the fix, I'm just saying that would be a huge part of fixing at least a lot of the oh, I'm to be a little fresh and say stupid mistakes. You know simple things. Where, and again in air quotes, an advanced attack end air quote was just basically, you had forgotten to remove an access, you know, and someone had an easy password. That is not an advanced attack and you see that all the time. When you have all these headlines and also vendors screaming advanced attack and when you look at what actually happened, you're like, oh, okay, okay, you know. So there is doom and gloom, sometimes by the marketing department of certain vendors and ambulance chasing by a lot of vendors and saying if only you were using our technology, this would not have happened. I'm like, holy moly, yeah, so, but I digressed a little bit.

Freddy Murre: 20:50

Going back to the cloud part, I think there's a lot of exciting things happening there where at least some of the vendors I've seen, as I said initially, that they're using the ability to scan every night and they update through all known vulnerabilities. When I saw that, I'm like, yes, now we're moving closer to at least providing value. Now, going out of CloudSpace again, is that some vendors, as I said initially, they're using the Intel function. They're saying, hey, what do you care about? Do you care about droppers? Do you care about these certain malwares? Do you care about fraud? Do you care about underground? Do you care about group X, group Y, what do you care about In which regions, in which languages, et cetera, et cetera.

Freddy Murre: 21:38

That gives a set of intelligence requirements so you can actually tell the vendor through the portal, or at least tell the portal, I mean, what you care about and filter out a lot of the noise. So you can say, whenever they pop up a report about this specific or from this specific forum that pushes a lot of credit card information or pushes certain malware or pushes, let's say, selling access, and if ever that triggers something on my trigger list or have these IPs or these domains, I would like to get an alert. So there is a way. But of all the vendors at least I've tested throughout the last seven years only a few, a handful, actually have that capability where you can say these are the things I care about in your reporting, these are, the rest is noise, I don't want it, right. So, and that's actually not what we're getting somewhere Right, but at the same time, I can then get a lot of reporting from one of these forums or from this certain malware, but it was used in a way or, you know, outside the campaign didn't hit our sector, for instance.

Freddy Murre: 22:53

So there is some massaging to do and, you know, checking and balancing the triggers you have, etc. So there's a lot of work to do on both sides, but there is some, at at least vendors that are doing um, there are certain vendors that is doing this the right way, uh, but uh, not too many yeah, no, but that's that's good to know.

Pedro Kertzman: 23:14

Uh, if uh customers can provide uh the requirements or the things that will matter most for them instead of just receiving any generic type of information, if the information is actually more relevant to them, I think it's a really nice thing to do. It just expedites the real intelligence at the end of the line. When it comes to intelligence, right, the different types of intelligence, the strategic, operational, tactical, the technical part.

Freddy Murre: 23:43

When it comes to cti, any thoughts around those differences, best practices, anything around that well, of course, I have a lot to say and, but we have, unfortunately, we don't have a lot of time, a lot of time to, so I'll keep it short. Um, so, my main job and I'm going to connect it into how I do cyber threat intelligence in my job at NFSearch. So I work at the operational level. So the order of things that I, or at least that we use, is that we have at the top we have strategic, then we have operational, then we have tactical and then we have technical. That's sort of the four levels that we work with. I'm not a tactical or technical intelligence analyst. I understand the results of malware analysis or log analysis or the things that the technical people do. I also understand the results of what the guys are doing in incident response, but I don't do it. I can't do it. So I work at the operational level and basically what I do is I translate the needs from the strategic level from the CISO, the chief risk officer, the chief operational officer. Those are the people I usually communicate with, or even managers who are outside of cyber and up. They don't necessarily understand or know as much about the things they shouldn't.

Freddy Murre: 25:10

In the old days, a CISO was sort of the technical whiz in the business. He or she knew a lot of the things. But more and more a CISO is now like do more than just be in the technical part. They do a lot of other stuff as well. Sometimes we get people like me who don't know all the technical parts, and that's fine. But my job, as I said, is to translate the results of an incident or a report into what does this mean for the business, right? So I work on a nonprofit. We have around 240 almost financial institutions across five countries with complete different cultures, different rules and laws. There's so many different things that I have to take into account. But what I've seen is the personas the CISO, the chief risk officer and other roles like that have more or less the same needs, more or less. So I can communicate well with them and try to translate, as I said in air quotes, what is going on, what this means to the sector, because I represent the finance sector, not individual organizations.

Freddy Murre: 26:18

They have to turn around and take the information I give them sorry, the intelligence and actually apply and use it in a way that makes sense for them in their context, in their organization. So they come to me with different types of questions. I tell them what I can and cannot do and, given the time and resources that we have, that's a good part of being the role I have. We call it intelligence engagement. Right, the stakeholder comes to me with requirements. I go through the process of figuring out what exactly is they need and want.

Freddy Murre: 26:52

By which time Can we do that at the expected level and quality with the resources and time we have? Yes or no? Is it within or without what I'm expected to do in terms of the mandate and the reason for me being? And then having a good discussion and explaining to them. You know, yes, we can do these things, but on Friday at noon I can give you this, but on Friday next week you can get 80% of what you asked for, and in another week I can give you 100% of what you asked for Because you know we don't have enough people or technology or resources available. Oh, I want you to do it anyway. Well, yes, but then I have to have this company come in and give me these consultants. This is going to cost you that. Oh, no, no, you're fine, I can wait for two weeks. So, having that discussion of what I want and what I need. So that's sort of the one part of the Intel cycle.

Freddy Murre: 27:51

That's sort of the intelligence requirements and stuff comes in there and that's what I work a lot with talking with these stakeholders and figuring out exactly what they want. Then turn around and talk to the internal team at the end of the circle five people there who do incident response and supporting the members. So whenever there's cases they communicate with, solve and help and work together with the different teams across all the five nations so I can translate the needs into something and they can then start working and collecting and doing their analysis at the technical tactical level, can then start working and collecting and doing their analysis at the technical tactical level, All the way from indicators all the way up to. You know, this group is operating in this way, using these malwares, with these droppers and these command and control networks, and they were usually publishing and traveling or active in these communities, etc. Which means because I've started challenging them well, what does that mean? I know, but I want them to articulate. What does it mean that this actor is operating in this way in terms of the natural or typical detection capabilities in the finance sector? Will they be able to detect how. Those are the things that we need to communicate to their peers, to the other CTI and other incident responders, across all the members. I have to translate this into the so what of? Well, what does that mean to the finance sector? Are we actually, are we threatened by how much? Know? What threat level should we set? What does that mean in terms of what should you then start doing in terms of planning for and preparing for something that might happen to us in two years or two months, or two, two weeks? So? So my job, as I said, is translating needs from the strategic level and communicate with the tactical technical level. They do their thing and magic. They send that back to me and translate again back into business speak. That's sort of what I do.

Freddy Murre: 29:52

Now there's this discussion, right, I think, Rob M Lee, for instance, and they say he and this other guy they say a CTI person should be able to do strategic, operational, tactical technical. There is no only strategic or only tactical CTI people. Well, I beg to differ. Most people I communicate with are tactical, technical CTI people. There's very few like me. I am not a tactical technical person. I'm actually paid not to be a tactical technical person. I'm paid to be a strategic operational person. That's my role, my focus. Do I do all of the things? Well, sort of Maybe 80% of my time is strategic and 20% I support. You know, with information and what we call basic intelligence, to insert responders and to the SOC and to other CTI people about certain threat actors and their behavior etc. But a lot of the people who do technical do not do what I do. I talk outside of their nearest peers and from the communities they came from.

Freddy Murre: 31:03

If we change the whole that we need to do everything, rather than saying that, I think we should be able to at least identify. When someone asks us a question on strategic intelligence, we should be able to identify it and then work on it or at least hand it over to someone who are more capable of doing it. If that's an internal resource or if that's a vendor or a consultancy, I don't care. But you, we should be able to service that to our stakeholders if they want it and if it's within the mandate I, why do we exist? So if they want me to communicate cti sorry, a tactical technical cti, or or intelligence, if you will to other technical cti people, then that's my role, then I shouldn't care too much about the strategic side right, but we should be able to at least do that initial. Why do we exist? I now we're into the stakeholder engagement part again. Why do we exist? I now are into the stakeholder engagement part again.

Freddy Murre: 32:04

Why do we exist? What are they expecting us to do? And do they even know that we could do more? Do they know and understand how to utilize and use us? And now we can bring in the Ferrari example again. Yes, we are the Ferrari in the garage. They can take us out for a spin and really demonstrate. We can then demonstrate our value. But if we do that on a dirt road rather than on the autobahn, then they're going to have a miserable time. We're going to have a miserable time and they're not going to trust us again. But if we can get them to use their Ferrari responsibly you know, don't drive too fast, Don't crash, Use common sense Then we can actually have a successful ride in a very expensive but also very high performance vehicle. I'm bragging a little bit what we can do in CTI now, of course, but I think we in CTI we can do a lot more than we do today, and especially with AI coming in, and that's a really interesting and challenging thing that we can discuss further down the line of this conversation.

Pedro Kertzman: 33:15

That's awesome. You touched briefly in a few aspects that I think are related to methodology. Would you mind if we drill down a little bit and talk about, let's say, the good and bad methodology around CTI?

Freddy Murre: 33:28

Of course. I mean we can spend a lot of time about the bad, but let's focus a little bit about the good.

Freddy Murre: 33:33

So in intelligence and in the intelligence communities across the world, there are standards. Some are written down and some are actually published. So in the United States they have what's called ICD-203. So Intelligence Community Directive 203. That's about doing high quality intelligence. If you follow those standards and there's more than 203, there's quite a few of them actually but that's a good starting point for most CTI teams and for anyone who wants to do intelligence, to start with those figuring out what it is that you can do with your resources and your sort of time available today and start adding a little here, adding a little there.

Freddy Murre: 34:18

That's sort of my biggest tip. Why am I saying it? Well, I spend a lot of time on conferences. I spend a lot of time reading blogs and being active on LinkedIn and communicating with a lot of CTI people and what I see is that there is a lack of standards in CTI in terms of the intelligence side. Cti in terms of the intelligence side. There are a lack of standards in how we communicate, how we structure reports. There's a lack of how we use words and how we use assessments, how we apply intelligence, tradecraft, ie structure, analytical techniques, et cetera. There's a big, huge mess, a pile of poop where basically everyone just do whatever they feel like and then call it intelligence, and nothing frustrates me more than people who claim to be writing intelligence were actually basically as I said initially today basically just pushing data or, at best, information. So trying to get a standard in has been really difficult.

Freddy Murre: 35:30

Now, since I started at Nordic Financial Search, there's been a change of course. I remember my first conference was the SANS CTI Summit in Virginia in 2019, january 2019, only months after I started, and what sort of struck me was that there was a lot of fluff, a lot of buzzwords, a lot of lingo, but there was not, as I said earlier today, there wasn't anything to back it up, right, there were no standards. Everyone was just doing their own thing. And we're still there. But they're slowly beginning to understand that, hey, every report should have a bluff, a bottom line up front. Why should I care about this report? Every report should have an introduction, sort of introducing the problem, introducing sort of why this is important to discuss. Maybe not to everyone, or I mean not to you specifically, but in general, but the bluff is to you.

Freddy Murre: 36:32

Then, going down, you know what is the reported information, what is the facts, what is that we know and what is it that we don't know? And be clear which is which. If you add any types of assessments, we've seen more and more actors trying to use what we call WEPS, words of estimative probability, highly likely or likely or even chance, etc. It's starting to be tested but not used in a very good way and also a lot of the reporting has usually maybe a bluff you know, introduction, these are the facts and then done. There's no conclusion, there's no assessment that says you know, based on what we're seeing, we assess that it's likely, that you know this threat actor will pose. You know that type of threat high level, medium because and here's the reasoning these are all the assumptions we had and this is how we covered these assumptions. These are the sources we used and this is a source summary statement.

Freddy Murre: 37:33

Basically, do we trust the sources? You know what's the quality of the sources and information? Do we have multiple independent sources confirming the information? All of those things? Right, because, to be honest, too often a lot of these reporting and blogs are trust me, bro, you know, I know because I did it. I'm like, yes, but where is the referencing? And that's sort of another thing. That's become a little bit better, but in 2019, I struggled to find references. Why should I trust you? Because you are this big vendor, are you nuts? I'm never going to trust anyone unless I can verify the information. In intelligence, we use the term you can trust, but you verify every single time, because people will try to deceive, people will try to manipulate. People will try to manipulate, people will try to do marketing gimmick rather than actually coming with the truth and showing and demonstrating being transparent with your work.

Freddy Murre: 38:35

There was this incident earlier this year. Again, I'm not going to mention the company, but a company had an issue. A cybersecurity company came out and said, hey, these are the things we've seen. The company, but a company had an issue. A cyber security company came out and said, hey, these are the things we've seen, and sort of pointed a certain direction with no references. Just trust us, bro. And a lot of the people in the community say, hey, listen, this is interesting, but you have to demonstrate to us and show what are you basing this on? Because we can't see it. The victim is denying that this has even happened and you know and you're showing us this sort of in air quotes proof, but we can't actually backtrack that information, we can't dissect it, we don't know where you got it from. And then they wrote another blog post trying to defend and describing the process as if you're trying to describe how to do intelligence rather than actually describing what they did. So, again, they created a big mess and they created more noise than actually reducing uncertainty and reducing friction, right? So these are typical examples not typical. It happens less now, but it was more, you know, uh, in 2018, when I started, uh, so, uh, it is interesting to see that it has been.

Freddy Murre: 39:53

There has been a lot of changes. So, methodology wise, there's more. When I go to first cti in summits in berlin, they have been in ber Berlin for many years now. I think they're going to change next year, but more and more same with SANS CTI Summit, more and more of the talks is about intelligence methodology, how to do. This is what I did, this is how we did something. These are the results we had, and there's more and more of it.

Freddy Murre: 40:23

And a lot of these summits also has a day of workshops. So the first CTI summit, they have a day of a workshop the day before and I've been teaching how to do intelligence requirement management to a full house for the last four years and basically what it is is identify your key stakeholders, identify what they want, translate that into some products that you can deliver. Go back to the stakeholder, ask are these the things you want? They say yes or nay, no, I mean. And then you adjust and fix it and then you're set with a certain set of products at a certain set of time, the key questions that they want. You start producing and then coming back to them and saying, hey, last six months, we're doing these things for you. What have you used them for? The answer usually oh, they were great. No, no, no, sir, you misunderstand. What decisions did you make based on the intelligence that we gave you? Ah, now you're getting into the core value of your actual deliverables.

Freddy Murre: 41:24

If you can't articulate and they can't articulate the decisions they made, the benefits that you supposedly gave them, then you are in a dark space. Whenever they want to cut money Right when they want to fire someone, or, let's say, they want to reduce their cost, they're going to fire someone. If they don't think that you deliver value, they're going to fire your ass. So your biggest job is to ensure that those who are sitting on top of that pile of money. Whenever they want to cut something or reduce something, they should think oh no, the CTI guy or CTI woman, they are actually golden to my work, I don't want to lose them right? That's a cunning way, if you will, of thinking of how can you ensure that your job is safe? How can you ensure through that, that your stakeholders are happy?

Freddy Murre: 42:19

Ie you understand who they are. You want to tailor your intelligence or products to their needs? Ie, you understand who they are. You want to tailor your intelligence or products to their needs? Ie they're going to work on certain projects. They need support on certain questions. Your products should be able to support them on that so they can make decisions. Understanding methodology of intelligence is actually really, really important. It isn't more important than doing the technical part, but it's equally important, because it's three words Cyber threat intelligence, not just CT or cyber threat. It's cyber threat intelligence and that's where a lot of people make mistakes. But again, if your core value, what you're expected to do, is support the SOC or support the incident responders, then by all means do it, but at the same time, at least ensure that you understand the needs of your other stakeholders and if they have needs, try to service them a little bit as well, so you can ensure that whenever there is discussions, they you are in their favor side and not their negative side. Does that make sense?

Pedro Kertzman: 43:27

that doesn't make sense in english at all, I realize, but you know the plus side rather than the minus side absolutely, no, absolutely, and I agree with you, even if you feel more comfortable or just serving the SOC or incident response teams, cti. In general, you can do more.

Pedro Kertzman: 43:44

And like you mentioned, serve upstream better the leadership with decision-making support. I think that's the key thing and I like the way you put it. If you don't validate they're actually offering value for decision-making, at some point you might get a surprise that might not be as pleasant. You know, when cost cut decisions come into place.

Freddy Murre: 44:08

But I realize that you know it might seem that I'm only talking about higher ops, but my point is cyber threat intelligence is supposed to support decision makers. If you make a decision whether or not IP is bad, if you make a decision whether or not email is bad, if you make a decision or not, if you have a certain vulnerability that you need to plug because this active campaign is trying to utilize that vulnerability, that's a decision. That is what we're supporting. We're supporting our peers, those who work in different parts, let's say the SOC, or their first line, second line defense, or if they're detection engineers or if they're threat hunters. We support them.

Freddy Murre: 44:56

Not just about the CISO, it's about decision makers. A decision has to be made and part of your mandate is to support a certain group of people. Who are those? That's what I'm arguing. I realized that throughout conversation I was focusing a lot on my job, but that's the example I use. The key job is to identify your stakeholders based on the role you have and why you exist, and then you have to figure out well, what type of decisions are they going to make and then support them with that to reduce uncertainty, et cetera, et cetera. So I guess that's a better clarification of that.

Pedro Kertzman: 45:33

Yeah, absolutely, and I think a very good example what you're saying. The decision making in general, not necessarily you know CISO or person A or B, and sometimes that might change depending on the scenario that the CTI person might be seeing. I think the most recent all over the news example that I think should be part of CTI conversations to decision makers is including now HR folks in the conversation with the whole North Korea. How can I? Yeah, suddenly we get a completely yeah.

Freddy Murre: 46:07

So we get a completely different audience that we have to communicate with. They have no clue of what we're doing, right, yeah, or we have to talk to a purchaser department because of they have, they're going to purchase technology that is either under some sort of law that you can't do, or selling that technology, or importing this, or that you have to sort of have what we usually say the grown-up voice. We have to sort of talk to our peers and communicate in the way that they understand to say, hey, but all of these audiences differ depending on the situation and we have to be able to communicate that Absolutely. Now, one thing I talked about was, you know, cti people usually have many hats. So I engage with stakeholders, I engage with, you know, insert responders. I do a lot of collection on my own. I do a lot of analysis. I I do a lot of collection on my own, I do a lot of analysis. I also do a lot of presentation, right? So in all of this I've covered basically the entire intelligence cycle, or the intelligence production or intelligence you know, process, if you will, and that's sort of the theory behind it if you will, you will always need some sort of requirement. So someone has some sort of requirement because they're going to make a decision. That's the core element. That's what we do in the direction phase, sort of the starting point of intelligence.

Freddy Murre: 47:33

Now a lot of people say, well, I don't agree, because when I saw that thing I thought that was interesting, so I sent it and they liked it. Because when I saw that thing I thought that was interesting, so I sent it, and they liked it. Yes, because you know that that's interesting. Ie, someone has a requirement. It's not that that person has to tell you about it, but through your job you know and understand that, hey, this is a threat, someone needs to know about it. That's the requirement that you're giving right. So it doesn't necessarily go from A then to B. It can actually be from B to A. And that's an important part about the Intel cycle that a lot of people say well, you have to do A, then B, then C, then D, etc. No, you don't. But usually we say that there is some sort of requirement. That is the starting point. You don't do a collection just for the sake of collecting. You collect on that vulnerability because it is relevant to you. Ie there'sa requirement. So these are the things that we're usually wrestling with when we talk about the Intel cycle, but I'll go into the cycle now.

Freddy Murre: 48:37

So first part is direction. Direction is understanding your stakeholders and the environment you're working in and your mandate, ie, why do you exist and what are you supposed to deliver on? And they come with a list of questions, and there might be multiple stakeholders. Some are important, your key stakeholders, and some are less, and then some are like meh, those are the consumers, but we can write stuff, but they can't tell us what to do. The client, customer, consumer level I talked about before. So once we understand who is asking for what, then we can start figuring out. Well, what is it that they want from us? Well, do they want the report? How often? Maybe weekly? They want it to contain certain things.

Freddy Murre: 49:22

All of these things has to tie back to decision. If it's like, oh, I would like an overview of recent ransom events, why, well, I want to know why? Well, I want to know because I'm doing this thing, why you ask the whys until you get into the core of why they want it. Oh, I'm going to make that decision. Bingo, that's what you're delivering on. If it's only about I'm curious, you smile.

Freddy Murre: 49:49

Then you go on Feedly or another vendor. Create a template and automate the heck out of it. Don't waste hours on this every week. Create an automation, get it out there and check every six months and verify that they're happy. If they're not reading it, try turning it off or delaying it for a few days. See what happens. If nobody cries, nobody uses it right. Don't waste time on stuff you don't have to work on.

Freddy Murre: 50:17

So, as I said, direction, direction. Who wants what? Next is what is it that I want, why, when, what time, at what quality, et cetera. Once you know who and what they want, and then how are you then going to deliver it? Right? Once you understand this, you can then start looking into well, what do I have to do to be able to deliver that product? Do I have to do some sort of analysis I have to do to be able to deliver that product. Do I have to do some sort of analysis? I have to write something. Do I have to write the blog? Do I have to write the report? Do I have to do a presentation? What should it contain? That's your production part, where you're. How are you going to create this thing that you're going to give to them that they can use for the decision.

Freddy Murre: 50:56

Now, for those of you who are familiar with Intel Cycle, you realize I'm going backwards. Now, for those of you who are familiar with Intel Cycle, you realize I'm going backwards. I went from direction then to dissemination. Now I'm talking about analysis. I haven't started collecting yet, right.

Freddy Murre: 51:10

So once I know who's asking for what and when, how are they wanting it, the dissemination what do I need to do to be able to generate that intelligence and what type of analysis I need to do? That's analysis and processing. Then I have to figure out well, what do I have of access have of data, what are my gaps in the data and gaps in my collection? What do I need to do to buy or build or gain access to either vendors or sources to get that data? And then you go back to your stakeholder and say, hey, you asked me these things, this is what I'm going to do. But because of lack of access or too short of a time or high demand and quality, I can only do these things or those things, or you still want it. Well, that's going to cost you X, y, z. Oh, you're fine anyway. Cool, then we're going to do it. Well, that's going to cost you X, y, z. Oh, you're fine anyway. Cool, then we're going to do it. That's usually what we say.

Freddy Murre: 52:04

The four steps in the regular term is you do direction, then, once you know that, you start collecting because you want to fill your gaps, then you do your analysis and processing analysis, I mean and then you produce, and then you do dissemination, then you do feedback. Usually that's the order, but to be able to do it, you go backwards, you go counterclockwise to do that, and that's what we call the intelligence engagement. We engage with the stakeholder to figure out their needs and then we discuss internally what we can do and then go back to them and tell them what we can deliver to them at what time, and then we execute, going the right way of the Intel cycle. That's the four steps of the Intel cycle.

Pedro Kertzman: 52:47

No, I love that. Yeah, I thought it would just be a natural follow-up from the previous topic. That's awesome, and you also mentioned a few times back then AI, right, how do you see AI changing the CTI landscape?

Freddy Murre: 53:02

This is interesting, right? First, we have to clarify a few things. Ai is a big term. Most people, when we say AI today, actually mean LLMs, large language models, chat, gpt and other competitors of them. That's what most people think of when we talk about AI. I think AI, automation, machine learning it's a huge, huge, positive thing for anyone who does intelligence and protecting organizations, cyber threat intelligence, incident responders, defenders of all levels right, ai is a big thing. We have been using AI for the last 10, 15 years with machine learning and automation and machine learning. Some would argue that automation isn't AI per se, but it's part of the machine learning part. I would say but at least machine learning, we've been doing that for the last 10, 15 years. So we are already slowly adopting AI into CTI or into cybersecurity, if you will. We've been doing that for a long time already.

Freddy Murre: 54:15

Now going into the other part, llms, that's a doozy because there are so many misconceptions of what an LLM is. An LLM is essentially just a text generation tool, and I want to emphasize the tool. Llms are tools, not the end product. A lot of people think that, oh, we are AI, the value of what we do is AI. No, you don't? You use AI as a tool to deliver the value that you're delivering, right? So using LLMs to write reports or to summarize reports, or to read a table of a lot of IP addresses and a lot of other things and use it in a creative way are all good. However, an LLM is a text generator, as I just said, and its main purpose is to generate text based on your input and the data it's been trained on. That's it. It will never know the difference between a finger, wrist and hand. It doesn't understand the concepts of and if you don't have fingers, it won't understand that you can grab something, right. It doesn't understand. If we're saying the LLM understand, we're actually saying it's sentient, it's aware. And the same with oh, the LLM understands, we're actually saying it's sentient, it's aware. And the same with oh, the LLM lied. No, a lie is a conscious decision to actually deceive someone. That means it is sentient. It isn't.

Freddy Murre: 55:54

An LLM basically is generating text. It's also meant to build text that resembles human writing in such a way that you will trust it, right. So that's what happens a lot. You see this summary of these three reports. You read it and go holy smokes, this is good and that's by design, right. It's generating the text based on the input. These are the texts.

Freddy Murre: 56:22

Now, when it adds stuff, that's not part of the text, that's not a bug, that's a feature. It's supposed to generate words. If it removes stuff, again you know it just generates text. It doesn't know if a certain point was important or not. It will not know whether or not that malware is more important to you or to anyone rather than that malware. It won't understand these things.

Freddy Murre: 56:52

And what I've seen, which is probably the worst, is when it combines things let's say, the number of fraud incidents in one report and the percentages of fraud activities in another report it suddenly combines those two into a new diagram that, on the face of it, looks really good, but when you know the data and understand the data, you realize that, hey, this isn't right. The data was there. It didn't lie or hallucinate, it didn't forget. It actually combined data that was in the report, but in the wrong way. So these are the three things that you should see.

Freddy Murre: 57:33

It either adds, it, either forgets, or it combines things wrongly and you never know when it happens things wrongly and you never know when it happens and that's the biggest issue right now with LLMs is that a lot of vendors are saying, oh, we have AI this or AI that or you know, and I'm like, well, that's good, but how do you fix the adding, the combining and forgetting oh no, we have guardrails in place. Meaning, oh no, we have guardrails in place. But if you do now that's my retort is if anyone has fixed that, you can be a gazillionaire right? Because according to those who know, those who are actually working with LLMs, they're saying that's not going to happen in a long, long time. What happens is they take all this information, the text, they train the LLM, they take out the LLM and that's a snapshot. And then they add guardrails, they add a lot of layers of information to guard, to keep the LLM in track in accordance with whatever purpose it's set up to be, with whatever purpose it's set up to be right. So you can have a medical LLM which will sort of try and help you with medical or someone that does the legal stuff.

Freddy Murre: 58:48

But we have all seen examples in the media where cases in multiple countries have been thrown out or actually won but then had to be sort of canceled afterwards because there was references to cases that were not real. There were a combination of legal terms and stuff that suddenly wasn't part of the documentation or they forgot certain things, right? So there's been so many cases of this. So going back to using LLMs for cyber threat intelligence is really good, but you have to understand the limitations. You have to go back and verify Trust, but verify is the key aspect here.

Freddy Murre: 59:34

Now I saw an interesting post on LinkedIn that asked you know. Interesting post on LinkedIn that asked you know, if I gave you a report that was written by a human, how much would you trust it? Okay, so people you know said this and that Good, if I tell you that this report was written only by AI, how much would you trust it? Oh, that was a completely different story, right? People went oh no, I don't know. People weren't trusting the pure AI product, which is interesting, right, that goes against what we're seeing. A lot of people trust the AI blindly that's what I see, at least. But when you ask them here's a human product, here's an AI product, which one would you trust more? People will trust the human product more.

Freddy Murre: 1:00:19

But then some will say well, you know, why are we keeping the ai to a higher standard than humans? Why? Why are requiring them to do an llm. I mean to be more correct than human and I don't have the answer to that, but at the moment the llms are, you know, 30 to 60 correct in some cases and 80 correct in some cases.

Freddy Murre: 1:00:46

But you know what never know when it happens. That to me is a problem, right, and if I'm going to generate a report where I don't know when the llm added, removed or combined stuff, I am not going to send it out, right? And also, going back to some of the things we said in the trade graph, you know what are my assumptions? The AI doesn't have assumptions. That's a human trait. Well, what are my sources? Well, the sources the LLM is using is usually blog posts and stuff, not the reports. If it uses the data set and report, it will still add stuff and you don't know when it adds, it forgets or combines stuff. So the whole interesting part is say, oh, I'm saving time. Well, are you? If you have to go in and reread everything and verify, you're actually spending almost the same time you could have while just reading the first time and then writing it.

Pedro Kertzman: 1:01:43

You're saying about no, that's, that's great. And you're saying about the linkedin and blog posts, any other places that you use apart from quote-unquote, traditional threat reports, any other places that are your like go-to places. You also mentioned conferences, or you name it books, yeah, any things you use to learn about more on the cti industry, I would say in general than this, you know, threat actor campaign or something like that yeah.

Freddy Murre: 1:02:15

So, as I said I, my focus isn't so much on the technical side but in terms of conferences. I would certainly try and go to SANS CTI if you can afford it, and the same with that's in the US, or the first CTI conference which is in Europe, usually in Germany. Those are really good for CTI people. You know CTI people on stage talking with CTI community and everyone there more or less are CTI people or CISOs or CROs etc. So it's a really good crowd to be there and to network, network the heck out of this. That's my biggest sort of tip of this. That's my biggest sort of tip In terms of books.

Freddy Murre: 1:03:05

I haven't found a lot of books that do CTI well because it's mostly technical and not so much on the analytical side or intelligence side. Some of them have some pages on it, but mostly it's more of the you know, sticks and Taxi and MISP and all of those things are more at the technical level and not so much on the analysis and the analytical level or intelligence level. For that you have to go to the intelligence community, the best book money can buy. It's really expensive, but it's by Fersen and Hoyer and it's called Structure Analytical Techniques for Intelligence Analysis by Fersen and Hoyer. That's sort of what we call almost the Bible, if you will, in terms of right information about intelligence, about analysis and how to do it.

Freddy Murre: 1:03:51

Well, and there's a lot of books by Fersen that I would also recommend. But if you haven't seen or read the SAT book, the Structure Analytica book, that's where I would start and then go online and look at blogs and other books and etc. To learn more about how to do analysis, how to do intelligence, and add elements of that into what you do with cyber threat intelligence. I've also created a huge mind map. It's called the Intelligence Architecture Mind Map. Put it on GitHub and it's available for everyone. So we'll put the link, I guess, in the description. Absolutely, and also teach intelligence Sort of a short plug for me. I teach structure analytical techniques. I do a lot of workshops, so just hit me up on LinkedIn and we can have a conversation.

Pedro Kertzman: 1:04:42

That's awesome. I appreciate it. Yeah, we're absolutely putting those links in the description of the episode. And any final thoughts for the listeners?

Freddy Murre: 1:04:52

No, I don't think so. Just go forth and do great things and work together with the peers and join communities and join sharing communities and share your insights as much as you can, because we all need to help each other elevate the quality of our work, but also share our knowledge in terms of incidents and knowledge about threat actors, etc. That's the best way for us to fight the increasing threat.

Pedro Kertzman: 1:05:23

Absolutely. I could not agree more. We have to share knowledge, because on the other side those guys are doing it. So we have to come together as an industry. That's for sure. Freddy, thank you so very much for all the insights. I really appreciate you coming to the show and I hope I'll see you around.

Freddy Murre: 1:05:41

Yeah, thank you for having me. I had a blast and looking forward to seeing you in person.

Pedro Kertzman: 1:05:46

Absolutely. Thanks again, take care, thank you, bye-bye.

Rachael Tyrell: 1:05:53

And that's a wrap. Thanks for tuning in. If you found this episode valuable, don't forget to subscribe, share and leave a review. Got thoughts or questions? Connect with us on our LinkedIn group Cyber Threat Intelligence Podcast. We'd love to hear from you If you know anyone with CTI expertise that would like to be interviewed in the show. Just let us know. Until next time, stay sharp and stay secure. We'll be right back.