John Doyle: 0:00

Everybody there is just the smartest person in the world and super humble.

Rachael Tyrell: 0:04

Hello and welcome to Episode 7, Season 1 of your Cyber Threat Intelligence Podcast. Whether you're a seasoned CTI expert, a cybersecurity professional, or simply curious about the digital battlefield, our expert guests and hosts will break down complex topics into actionable insights. On this episode of Season 1, our host Pedro Kurtzman will chat with John Doyle, who has over 16 years of experience working in CTI, digital forensics, cyber policy, and security.

Pedro Kertzman: 0:57

John, Thanks so much for joining the show. It's great to have you here.

John Doyle: 1:00

Hey, Pedro. Thanks for having me. I'm really excited about this.

Pedro Kertzman: 1:02

Awesome. Usually, I start asking the guests about their journey into CTI. Would you mind walking us through that,

John Doyle: 1:11

Yeah, for sure. My story, I feel like, is comical, much like a lot of my colleagues in this field, how we almost got here by accident. My journey kind of starts at university. I was working... 2008 timeframe for our Geek Squad, or the equivalent of our Geek Squad on campus. And at that point, I had spent a few years, and that really kind of got me into computers a little more than I had previously. So there was that dynamic. I ultimately went to grad school for information security and also picked up this degree in national security studies, not realizing that kind of would position me well for understanding the geopolitical overlay of events that happen and how there's cyber actions that are interrelated. Spent a decade thereafter working for the CIA as a threat intelligence analyst focused specifically on cyber threats. And then I eventually made my way to the private sector. And in the private sector, I was working and working for Mandiant and I also moonlight as a SANS instructor for the threat intelligence course. So I've kind of I've kind of gone an interesting route, one that I feel would resonate a lot with people who used to work for government, who kind of came out and transitioned into this field, and one that I think is becoming a little more non-traditional in the past decade or so, as the really juicy data for threat intelligence analysts no longer lives in government spaces and classified sources, but actually lives in victim environments that incident response firms and managed defense providers or MSPs actually have access to.

Pedro Kertzman: 3:00

That's interesting. I always thought that still Nowadays, military intel experience would be super desirable, for example, on the CTI field. But it's interesting to know this shift. I never thought about that, but it makes sense.

John Doyle: 3:19

Well, there's two parts to it. I think you hit on an interesting first bit there when you talked about the skills. So one of the things you get in military intelligence or even civilian and law enforcement intelligence is kind of grounded tradecraft. So you've got to be able to communicate. communicate well, you got to be able to do research well that can be thoroughly backed up based on this or that data source. So you learn kind of the basics of research and it gets reinforced and then you learn how to communicate it over time to stakeholders. So you're learning analytical processes, which is incredibly helpful irrespective of what job you have. But the cyber element is what's kind of or at least the sources of information is where we've seen kind of a change happen in the last decade.

Pedro Kertzman: 4:06

there you go yeah the other day i was talking to somebody that they mentioned uh something like that uh the intelligence knowledge should come first than the cyber knowledge you can combine the cyber into intelligence but if you don't know how to do intelligence and you just have the cyber knowledge then you kind of need to learn from the get go a new trade craft because intelligence again even though we leverage cyber security uh if you just do for example, malware analysis, but then you need to learn how to connect the dots into a more intelligence-driven type of approach. Does that make sense?

John Doyle: 4:43

Yeah, I can see that. I can most certainly see that. I think, I mean, if we back up a little bit and look at almost the start of CTI, you had two different audience types and two different skill sets that we were trying to speak to. So you had the incident responders and the SOC analysts doing kind of threat intelligence organically to make a determination as to like, is this bad? Should I take an action on it? Does it require immediate attention? Can I build out a playbook? But that's different than speaking to like an executive level audience where they're looking at a more strategic overlay. They're trying to identify complexities and trends in cyber threats and how this relates to their organization's risk posture. And we're actually seeing, definitely seeing kind of more alignment happen and more convergence happen over the last decade. But it's just two different skill sets. I mean, on the former, you've got kind of a digital forensics incident response background, cybersecurity background, moving into a more higher level service support construct. And from the other side, you have traditional intelligence analysts who are used to providing insights in a way that resonates with different audiences at a high level so they can make informed decisions who are then having to learn the more technical and the more nuanced aspects of this field.

Pedro Kertzman: 6:15

Got it. John, awesome. And so you mentioned a few aspects, important, very important functions within a CTI team as a whole. Any other changes that you've seen in the past possibly few years within how those roles are evolving or changing based on either the maturity of the industry itself or from external factors, for example?

John Doyle: 6:47

Man, you asked a loaded question there. Yeah. Let me think about this. Yep. We've seen... a few different evolutions. So there are more companies that have access to firsthand information and also secondhand information that is really important for building out an understanding of adversaries or threat activity groups. So on the vendor side, there's more vendors with useful data that exists that is happening concurrent to a high level demand for threat intelligence to help inform decisions whether it's at the board level the executive level the cyber security program manager level or even the threat intel manager level then there is the skills so the demand from other internal stakeholders is then predicating the need for individuals on the teams to be broader and deeper in their specific craft. And that's causing, depending on the organization, the resource, the budget, all of those kind of administrative constraints and kind of growth projections, causing individuals to have some level of specialization, but also having the need to have a baseline skillset or at least a functional understanding of how what i do compares to what you do and how it's complementary and how we can then use that together to work effectively to drive better outcomes

Pedro Kertzman: 8:32

got it okay that's uh that is interesting you mentioned like part of the Evolution now, the way I interpret it, it's bringing more value to all levels of the organization and not anymore just the SOC or just that particular department. Actually, I was provoking people the other day that threat intelligence is not just threat feeds and reports anymore. You can have way more emission decision making. You can use that CTI information to do way more than just playbooks with feeds and understanding what's going on with the reports and things like that. So it makes sense. It's interesting and it's good to see the industry overall shifting in that direction. One of the struggles I still hear a lot from CTI folks is being able to prove the value of their department into the upper management levels. So they don't get as impacted or sometimes with layoffs and stuff like that, but even, hey, I can help you with this. If you have like a decision to do, I can bring Intel and maybe we'll quote unquote, enrich the data that you have to take that decision. Sorry, go ahead.

John Doyle: 10:09

No, no, no, no. I mean, you're hitting on two things I think a lot about. One, what's the best value proposition for an internal CTI team? And then what is the value proposition for intelligence vendors who may service internal CTI teams? And this is something that when i'm working with clients who are building their cti programs or trying to mature it these are all decisions that they're trying to make these are all complex situations they're trying to navigate is how do i convince the business that we have the ability to provide proper level support to service these needs that are measurable that actually show impact both at the bottom line of the company, but also shows efficiency gains. So it's... My boss at one point in time said something to me, and I don't think that he meant it as like this wise sage advice that was super impactful, but it actually was. He's like, look, look, John, like organizations use CTI for like one of three reasons. One is you've saved the company money and money could translate to like brand reputation loss or a whole host of other kind of things that are associated with risk. It's making the company money or you're improving efficiency in some way, shape or form. I was like, damn, that was good. I need to, I need to take more mentorship from you.

Pedro Kertzman: 11:42

Yeah. Mentors. Right. So, so valuable. I feel you. Yeah. Some quotes just have that power to stuck on our heads and we will keep thinking about them and using as a guideline to do, you name extra work research, something like that.

John Doyle: 12:00

There is like one more thing here. Have you heard of the CTI-CMM before, the CTI Capability Maturity Model framework that exists?

Pedro Kertzman: 12:11

I heard of it, haven't had the chance to deep dive into it yet.

John Doyle: 12:18

Okay, I'm going to give it like a 30 second plug here. So it's a model that's designed to effectively demonstrate that value prop across different stakeholders. So here is incident response, here's SOC, here's hunt, here's red team, here's third party management, et cetera. What is there? organizational function do and how can CTI support it? And then it breaks it down over a spectrum of maturity levels. What they just done in the past month is actually created metrics. So I'm an internal CTI team. What are good metrics for me to measure against if I am supporting incident response, if I am supporting purple team, if I am supporting red team exclusively to help with, because it's hard, right? Metrics, creation, is not something that like me as a deeper practitioner have been trained to do, or even like somebody coming into a management role. They've not gone to, you know, university or taken professional training on like, how do I create effective metrics? But my boss is asking me to create something, so I have to create something. so this framework is actually designed to bridge a lot of these gaps to help with that value prop for intelligence teams as a whole so like if i didn't have that organically i don't have to create it from scratch it now exists it's something that i can take and pull from

Pedro Kertzman: 13:41

that's amazing that's that's super important to how to to sell the value of your of your cti team uh and maybe um i don't know brainstorming moment um do you think cti could go uh and offer as well value to um some companies used hr to manage uh cyber security awareness program or you know work handy hand with it for that but do you think for this type of program cti could also offer some value

John Doyle: 14:22

absolutely I think the closest partner that a CTI team could have besides incident response is security awareness. And that's for a host of different reasons. So one of the countries I used to track a lot was North Korean cyber threats. So all of the groups associated with that. And in the last year, One of the trends that we started to see emerge, and it's not net new, but it's kind of net new in the public's eye, is this notion of North Korean IT workers applying for roles at bespoke companies or even big name companies, which has fundamentally done something quite unique, I think, in this field. It's allowed us to not just look at threat activity. In part, some of these actors are using their access for threat activity. or providing the access remotely to cyber operators. But it now lets us actually bridge the gap because this is something that HR needs to know about to be able to screen out North Korean IT workers that are trying to work for their company. It brings in the notion of insider threat and it really kind of expands the aperture to show what that value prop could be for threat intelligence to security awareness training, HR. identity and access management and like a few others and it's that's not the only case too we've been seeing but we've also seen like i'll admit i was a bit of a naysayer when it came to you know using the dark web to surface things that are useful for cyber threat research But like the notion of identity intelligence is yet another area that's starting to prop up. So leaked creds, customer PII, internal PII that's leaked, like all of that's really important. And I think this just speaks to the evolution of the industry in the evolution and almost remit in things that are expected of a CTI team to be able to cover, let alone knowledge of like exploitation of edge devices and some of these other things that were more like, whoa, that's very unique, specialized knowledge, but now it's almost expected.

Pedro Kertzman: 16:37

Yeah, no, that's great to, let's say, brainstorm that with you. I'm going around some local conferences here to try to give real use cases and thought-provoking things to people that CTI should go beyond just threat feeds and reports. you're basically short selling your value if you're just focusing on that, right? So it's great to hear that I'm not alone. Yeah, no doubt about that. That's awesome. Man, and you mentioned about the CTI maturity model as well. Any other interesting frameworks that we've seen around that perhaps not everybody is using and they should use more or explore more? Anything around that? N

John Doyle: 17:32

I'm a little biased for what's going to come out of my mouth as the author of it, but I'm going to do my best to give it as an objective of a sell as possible.

Pedro Kertzman: 17:43

Fair

John Doyle: 17:43

enough. So a few years ago, we had been getting asked a lot of questions on the Intel consulting side about, well, what's the right composition for a CTI team? What are the right skills? What are the right backgrounds? Who should I hire? How should I hire? It's all sounding very familiar, isn't it? So I kind of went out there and I'm like, well, let's see what NIST has to offer. NIST is a standards body. They do a pretty good job across the board at putting useful information out there. Let's see what they have. And I came across NIST SP 800-181, which is the NICE framework, the National Institute on Cybersecurity Education. Oh, cool. Fantastic. Let's drill down into this. And as you start to see all of the different role profiles that would be part of a cybersecurity program, CTI gets parsed out almost into three or four different categories. You've got a collections manager, you've got a threat warning analyst, you've got so and so forth. And as I looked at this, I was like, I'm just curious how much government influence there was here. And whether there was anybody from private sector who actually weighed in on this. So, you know, I ended up connecting with two individuals who were leading the NIST NICE project, and we had a very cordial conversation. It was a very good conversation. And they're like, yeah, you know, you're right. A lot of this was government-backed. Like, we kind of leveraged them for, like, what does right look like? I was like, cool. I work with threat intelligence teams that are, like, six people max you mean to tell me that one of their six people is just managing collections and like that's all they're doing now I get it if you're working in like a several hundred person intelligence community that these are distinct roles but you mean to tell me that there's an expectation being made here that like one person is doing just this job exclusively like yeah okay to each their own um So, so really kind of prompted me to come back to the table and I asked her, I was like, well, I've got a lot of ideas here from what I'm seeing. Is there a way we might be able to like merge this from like your knowledge and skills and abilities you've broken out or your KSAs you've broken out across the different, um, the different role profiles. And they're like, yeah, but we're actually thinking we might switch the model up a little bit and it's going to actually potentially break things. So like, if you want to go out and do something on your own and create your own framework, that'd be, that'd be cool. Like we'd love to collaborate or at least have insights. I was like, yeah, I could do that. So this led to the creation of the Mandiant CTI Analyst Core Competencies Framework, which is broken out into four different pillars. Those pillars functionally can be broken into two categories. The first category is soft skills and professional effectiveness. And the second category is technical acumen and threat knowledge. So we enumerate 182 different knowledge and skills and abilities inside of this framework itself, where if you're interested in just joining the field or like you've been in it for a while and you're like, all right, what's next? You could actually rate yourself. So there was kind of three different cruxes for why we created it. The first was self-inventory evaluation for personal development and professional development. Second was to help organizations with hiring decisions because it's like, man, what do they need to know? Like, I feel like this guy just needs to know more than the MITRE ATT&CK framework and what it's used for. Like what are those other things? So it was designed to allow organizations to almost lift the respective KSAs to include in their job requirements. And then we had a third one too, which I kind of bled into a little bit already with the description of the other two, but it's for that evaluation. How do I evaluate where my staff are and how they can grow? So creating almost like a team report card across the different areas based on the role profiles. So that's one that I think I think it works quite well. It carries the Mandiant name because I was being paid while I developed it, but it was done in coordination with the private sector. A bunch of my peers who are in the industry here, either at vendors or working simply in private sector and also in public sector too, they were able to weigh in on it. So that's, I think, one of the frameworks that maybe got me invited to work on this cti cmm project okay um so it's it's been good i mean the the the genesis of a lot of these projects is to help fill gaps in industry whether it's showing up knowledge or helping programs build to better because we're all in this together right it's a small industry

Pedro Kertzman: 22:42

man well yeah that's great let me uh digest a little bit here and pause for a second. I like it. And by the way, just mentioning about the Sense CTI conference, it reminded me, any other good CTI-related conferences you had the chance to attend? Any values you saw on this or that other one? Anything to comment on that?

John Doyle: 23:15

I love this. I've been thinking a lot about this too. Yeah, there's a whole handful of them and I think it speaks to kind of both the specialization and the growth of the industry in different geographies. Some conferences are better suited for generalist audiences or those who are kind of entrant into the field, maybe upwards of three, four years. Some are better for researchers who have been around for a little while longer looking to connect. And conferences usually take one of two forms. One, it's open and you pay to get in, or sometimes it's free for live streaming. The other is kind of like this closed trust network where you need somebody to vet and verify who you are and that you're trustworthy because information being shared might be shared at a certain sensitivity level like TLP Red. Only the people there can talk about it. Maybe we start with the SANS CTI conference since you already mentioned it. That one happens in January. It's in DC. DC, Alexandria, Arlington area, usually. It's been a good one. It's been one of the longer standing ones that exists. Free to stream, nominal fee to show up there in person, usually maybe about 250, 300 show up in person. It's a really good way to network. And maybe I back up for a second. There's different value propositions for people who go to conferences at different levels. And it's a little weird too, because I feel like a lot of us on this field, we tend to be introverts. So like our social battery is drained and like going up to somebody like thought of me and like, hi, I'm John, who are you? And what do you do? And who do you work for? And like the 20 questions, it's just intimidating.

Pedro Kertzman: 25:03

I'm happy to know it's not me. It's only not me. Oh man. I go back to the hotel. I'm like crashing, crashing.

John Doyle: 25:10

Yes. Yeah. No, but like, that's, that's normal. And like, that's, it's funny. Cause my junior analyst, I talked to a lot and I'm like, you just gotta put yourself out there. Like at some point, everybody like we have this shared frame of reference where it's like shared experiences so like if you're being a little socially awkward guess what a lot of us are socially awkward so it's fine but we're in our own heads most of the time where it's like i don't know but what if i said what if i sound silly and i say something that i shouldn't like whatever

Pedro Kertzman: 25:37

everybody

John Doyle: 25:38

like we're our own worst critics so it's fine yeah So the SAN CTI conference is really good for actually learning. They actually did a split track last year and the year before where they had kind of like new to cyber. So people who are kind of generally in that first three, four year bucket. And then they had like a not like they didn't call it anything special, just like this is like the other track that we have. It's kind of cool because I saw a lot of individuals who were in person who had been seasoned practitioners for like a decade plus stop in for some of the new to cyber talks and they have a whole Slack channel where people can ask questions and bounce ideas off of each other. Like I saw some of the more seasoned analysts and researchers actually answering questions that were being asked during the presentation in the Slack channel. So it's like a way to kind of help mentor and build people up. So I'm actually a big fan of any conference that has a Discord server or a Slack channel that's already kind of pre-established to allow for that kind of growth amongst peers. So that's, that's San CTI. There's like a few that are coming up soon. There's a cyber crime one called sleuth con that's coming up in June. It's not the only cyber crime or underground type economy one. So like team, team comry rise, it's in three different locations to cater to different, uh, analysts in different geographies. One is in, um, Singapore for the APGA region. One is in Europe and one is in, I believe California. And they happen, um, you know, at different, times throughout the year. Those are good ones. You've got SleuthCon, which is cyber crime focused. I talked about that. The EPT or nation state version of that is CyberWarCon, which takes place in Washington, D.C. in the fall. But there's more. There's a whole host of other ones. So for the EU segment or even the UK segment, I'll kind of include them as adjacent and combined for this. You've got cyber threats, which the UK NCSC puts on. You've got Virus Bolton, which is actually taking place in just a few weeks. You've got First CTI, which is usually hosted in either Berlin or Munich. Then you've got TIX, which is out of the Netherlands, the Threat Intelligence Exchange. And I feel like I'm missing one or two there, but I feel like I hit on most of the high-level ones. Oh, and then there's some of the private closed ones, which include like LabsCon and PivotCon. And Woo, and maybe a few others.

Pedro Kertzman: 28:20

Cool. And any of those conferences... Would you say it's like a must-go for seasoned CTI guys? Like a must-go if they don't know yet?

John Doyle: 28:31

Yeah. Cyberwarcon and Sleuthcon... Sleuthcon's kind of new. It's only been around for a few years. It used to be Brunchcon, because it was the day after Sleuthcon, but then it got kind of parsed out as its own thing. You end up meeting... a lot of really interesting people at both of them who have been practitioners in the field for a very long time. And I mean, one of the things that resonated with me at both of those conferences, and maybe this is more representative reflection on the field, is like everybody there is just like the smartest person in the world and super humble. Everyone's willing to talk to you about anything and help kind of mentor and grow. So the networking dynamic you get from either of those conferences is wild.

Pedro Kertzman: 29:21

That's so nice.

John Doyle: 29:21

But of course, the drawback is you're in the U.S. around Washington, D.C. So for some people, that's cost prohibitive if they're trying to travel internationally, which is why I really like the evolution and advancement of these kind of EU-type equivalent conferences to make it more accessible, this field to be more accessible. Because you still get some researchers who speak at the conferences who come from the U.S. or will come from EU or will fly in. So it's not like there's diminished value in any of them. It's just they're kind of sometimes different flavors of the quality you get. But the people who go, top notch across the board.

Pedro Kertzman: 30:06

Yeah, that's a great point. I think the audience will change, but the quality of the speakers might be similar, especially for the conferences paying for flights and stays and all that. So, you know, if people want to go there and have their session or keynotes, you know, it's easier for them. But it's not as, like you mentioned, prohibitive for people. thousands or hundreds of people flying from one continent to to the other yeah i think black even black hat now has like a apj uh europe and and us of course

John Doyle: 30:41

yeah and if you're looking for kind of more local cyber security conferences so if we extract a little bit outside of cti the b-sides conferences are great to go to

Pedro Kertzman: 30:49

oh i love

John Doyle: 30:50

them there's local chapters of these sites just I mean, throw a rock in any which direction, you're going to find a local chapter.

Pedro Kertzman: 30:58

Oh, yeah. Honestly, I think any mid-sized city in North America, I would say, probably has a B-Sides at this point. Or a metropolitan area. Yeah, yeah. Awesome. Yeah, so changing gears a bit, you mentioned you used to work for the CIA. Of course... you know, details are completely confidential, but generically speaking, any specific part of the job, how used to be like any part of the routine you could mention and, uh, any insights from that, um, and, and maybe from a broad audience perspective, things that could, uh, people could leverage from the overall, uh, industry.

Unknown: 31:46

Yeah.

John Doyle: 31:46

Yeah, no, that's fair. I'm not sure that I have any that I can publicly share, but there's a lot of things that happen behind the scenes that do make their way into the news. And likewise, the news cycle helps prompt some investigations into different actor activities for analysts or analysts. So we're always looking for things that are related to whatever our focus is. So open source being one of them was absolutely something that I was looking at. And I was like, oh, this group is saying this. And then for us, maybe there would be a policymaker in Washington, D.C. who would read this particular news thing and ask a question like, hey, what does this mean? Should we care about it? And then the question would filter its way down to the respective organizations and we'd answer it. We'd have the opportunity to kind of weigh in, add some ancillary information about it, maybe the actors, maybe the groups, maybe the types of activity, and not just help them with the immediate ask, but also anticipate what other information they might need. And this relates to this trend that we're seeing, which then allows for some opportunity analysis on top of that to help make life harder on threat actors, or clamp down on threat activities, or help guide you know, The funding streams of different initiatives or just otherwise kind of highlight gaps and areas where the policy community could plug in to really help fill them. So in a lot of ways, it was kind of a really cool environment that I found myself in that I didn't think a defer practitioner had that type of a flair or could have that type of a national level impact. But yeah, kind of here we are. It is unfortunate that the agency is very close-held about some of their successes that come out because I think it'd be really cool for some of the stories if they did make their way public.

Pedro Kertzman: 33:54

Yeah, share best practices. I would imagine things of that nature, right? Yeah, it's a complicated trade-off, right? Because you don't want to expose those best practices because people can, on the other hand, could leverage that to maybe bypass them. But it's interesting to think the line, right? Where's the risk of sharing that, but also the risk of not sharing that with a broader audience that could, I don't know, become targets, for example. So it's a tough decision to make, I would imagine.

John Doyle: 34:37

Yeah, so on the one hand, the stories about what was worked and what the results for the impact might not come out. But the tradecraft, I think, is coming out for threat research because we're seeing, and we have seen for the last decade plus, a lot of really talented individuals leave government to go work for private sector vendors or individual companies. Then you've got the ISACs or other trusted communities or these conferences where it's opportunities to share best practices in how we go about doing tracking, how we do alignment to stakeholders, how we find signals in the noise that are interesting and build that intuition. There's I feel like a lot of that is out there. A lot of it's becoming more public knowledge, certainly a lot more than like a decade ago or more, because of the high prevalence of people who have rotated out from government space, military intelligence, civilian intelligence, law enforcement. And honestly, I think that's probably done in insurmountable amount of good for building resilience for building cyber security for a bunch of organizations that would otherwise be victimized um that at the time was probably scrutinized like you're leaving us to not work on the mission you've lost like you you've lost your your vision you've lost your perspective it's like nah not really i'm still getting mission impact so Overall, I think what was a bad news story for governments with the attrition turnover actually turned out to be a really good thing for bolstering this industry and bolstering cybersecurity resilience for organizations to help them bounce back and detect ransomware to help. to help foster public-private sharing, to help grow the overall security posture at the national and at the economic level, right? So, I don't know. You got me on a rant there.

Pedro Kertzman: 36:59

No, this is actually an excellent perspective. Thanks, John. I appreciate it. It makes total sense. I think at the end of the day, everybody is benefiting from that perspective. pool of amazing resources that the government in some shape or form was able to provide and train. You know, like you mentioned before, right? This is a team sport. We can only win this together. If you have like a super knowledgeable, insane expertise, part of the equation let's say the government from municipal states uh federal level but then the companies you know are falling apart from a cyber security standpoint then you start getting problems with the infrastructure manufacturing and then as a whole it's just not going to be positive to to anybody right anyhow um You mentioned Isaac, and maybe piggybacking a little bit on one of our previous topics, do you see the Isaac role, quote-unquote role, is also evolving? Some of them, or are they... primarily focused on sharing the more traditional forms of CTR, like feeds, reports, or you see the role of the ISACs also changing over time?

John Doyle: 38:45

That's a tough one. So when we say ISACs, what we mean is information sharing and analysis centers, ISAC being the acronym for it. Every industry kind of has their own ISAC or ISO. For our purposes, let's just use the word ISAC to avoid technical distinctions. Every industry owns and operates the ISAC model just a little bit differently. So the standards and practices in play The membership terms and service for the healthcare ISAC versus the retail ISAC versus the financial ISAC versus the IT ISAC, you know, so on and so forth, are all going to vary. So you'll have different standards levels for each. I'm seeing, so I work a lot with the Healthcare ISAC and a few others that I won't name, to really kind of help bolster their capabilities. And you're right, historically it's been, well, maybe we just share out a bunch of IOCs. Some of these ISACs are actually looking as part of their membership criteria for the participants, the industry partners that are operating in that sector to share with them threat insights to then share out at a sensitivity level. So whether that's TLP Amber or TLP Amber Strict that says us members here, across organizations, we can use it for the intent purpose of hunting this type of adversary activity, or because we're trying to determine whether or not this is a campaign. So something has come in, a tipper has come in from an industry partner, and they say, hey, we're seeing this proactively. Is anybody else also seeing this type of activity? So in a lot of ways, the ISACs end up being a nice early warning system to determine whether there's a campaign of significance, and chances are that that information is probably also being shared in other closed channels too. But there's this push and pull model. This push and pull model, and I'd actually go out on a limb and say that we are seeing an evolution over time for getting vendors and other researchers in there to help brief the ISACs at like their annual or semi-annual conferences up on tradecraft. Or what does best practices look like? But that's going to be... ISAC to ISAC, so I can't make a general statement there because of the way they're owned and run. But I have been pleasantly surprised in the last few years with the ISACs that I've been working with to see them grow, to see them take on board and try new things or have a centralized threat intelligence platform that can act as a push and pull model that has reports that are provided with those different classification taglines to allow them to be able to share things back and forth

Pedro Kertzman: 41:48

awesome and and john one of the things we hear probably every single day now many times it's uh ai uh you see any um kind of good usage or shifting uh across our industry if you will uh when it comes to the usage or where ai could help us uh um on any directions you can think, from a detection standpoint, LLMs, you name it, any sort of AI over there helping the CTI industry?

John Doyle: 42:28

Yeah, maybe I'll start with a quick story. So I was asked this about a year and a half ago by some colleagues. We were brainstorming, and then again by a client. And then at a workshop, I kind of put on my slide that says, AI. What is it good for? And then I just put up a bunch of different high resolution fantasy images. And I go, it's great for creating characters for your Dungeons and Dragons campaigns. And it got a good laugh. But it's true. There's a lot of like-minded or like culture type of like nerd culture here where we've all got our own kind of hobbies and things. And image generation is absolutely one of those. Go to find out with things like Mid Journey and Claude and other kind of models there. But as far as CTI specific applications, applications. If you look at the intelligence lifecycle, all five phases of it, there is application for AI in all of them. And when I say AI, I'm specifically honing in on AI is a tool that's used by humans. AI is designed to augment our capacity and our capabilities. So it is something that's used not by itself necessarily, but to help us with things like data triage, like looking at large scale data sets or leaks of data and being able to surface insights pretty quickly, a lot quicker than the human would be able to. It helps us with standardizing reporting. I use AI almost every day for writing things like, actually, I just used it yesterday. I said, hey, I need to write a pair bonus for this guy. He did the thing. Please include this, this, and that, and put it in a formal tone that highlights these different attributes that I know are going to be useful for helping maybe get him promoted. It spits something out for me in about 30 seconds that otherwise would have taken me 15 minutes to write. And even if it gives you the 80%, it's a really good starting point. When you start layering on top of that kind of rag, the augmentation for it to go outside of its closed data sources and go doing deep research, it really kind of shows a value add prop. Now, kind of the drawback with that is taking anything that the AI gives you as truth without doing vetting and validation. So for us as analysts, I feel like a A lot of more senior analysts are using this almost like a search engine, like on steroids. And a lot of the more junior analysts are just taking it and saying, all right, this is great, we're done. So like that critical thinking and that trust but verify mentality is quite important. But as far as producing information in a quick way, doing triage, outputting information in a standard structure, like please extract all of the IOCs from this particular report and map them to MITRE ATT&CK. it does that pretty quickly. Oh, and please put that in a tabular format so that I can take this table and copy and paste it into like a spreadsheet or give me this in a CSV file. It's really good at doing tasks like that. Things that otherwise would have taken us a lot of time to do the manual curation of and transformation. It can do that pretty quickly.

Speaker 01: 45:51

I've

John Doyle: 45:52

used it a little bit for the vibes coding scripting. It works fine enough. I'm just right. I'm not a developer. I can write a few scripts. It can write the scripts in Python a lot better than I can though. Anytime I need to do a parsing exercise or I need to link datasets together, I always just take it at face value and evaluate like, is there anything here that I need to change? I will review the code base, but ultimately it's still the human in the loop. I don't know that we're gonna lose the human in the loop, but boy, are we certainly becoming a lot more productive as a result of it. So it's cool because in a lot of ways, AI is letting those resource constraint CTI teams of like two or three people really operate at the level of like a five or six person team. If it's used right, if it's allowed in the environment based on the risks that have to be accepted for it, then there's a whole host of other considerations too. But at the same point, it's helping kind of bridge the gap and allowing us to support these different stakeholders in a way that historically we were resource constrained and couldn't. Whereas today, it is really kind of transforming the way we do intelligence. as a technology enabler but not as a replacement if that makes sense

Pedro Kertzman: 47:14

100% I agree with that it brings me to a point whenever we're trying to let's say learn more about CTI and I'm not necessarily talking about the feeds the latest threats IOCs and reports and so on You have any go-to source to learn more about the industry in general outside of conferences or new frameworks out there, things related more on the CTI holistic approach, any go-to sources for it?

John Doyle: 47:59

Katie Nichols and Andy Piazza, both independent of one another, pulled together to blog posts on their medium sites. On their medium sites, it's something like the Newcomer's Guide to CTI, really designed to provide a lot of resources on, well, what does seminal things look like? What are some of these key resources to understand and really get into this field? Less so on what CTI does, but that's implicit and kind of covered indirectly as part of that. There's also... the SAN CTI Annual Survey. Are you familiar with that one? No, I don't think so. So it comes out every year. They've been doing it now for a handful of years. It really gives good perspective not on the vendors, but on where industry trends are coming and going. So it is volunteer by nature. They usually put like a two, two and a half month call out to get data for people to take the survey to fill in, you know, exact thing you would expect from it. Like what industry are you in? Like what's your, you know, average years in or whatever. And like, capturing some kind of meta statistics that could be used for vetting. This year's one, I think, is due to come out in about a month. I would be surprised if we didn't see the inclusion of AI in it, if we didn't see the inclusion of these different like intelligence sources and vendors coming out. So like the identity intelligence, for instance, like that sounds like something that I've been seeing kind of the, you know, Intel 471s, the flashpoints and others really kind of digging into dark web more and having like the recorded futures, like add a dark web module or Palo Alto's like add that as part of their feeds. So like, I would be very surprised if that, know wasn't a trend that they tried to capture in it too so like what are those data sources being used by internal cti teams beyond like internal internal telemetry um so it's it's really good for kind of providing a lay of the land and i think that's what um that's where i would go if i'm trying to get kind of holistic capture at least a quick snapshot of what the industry looks like

Pedro Kertzman: 50:16

Oh, awesome. Great information. Thank you. Make sure people listening that you check those medium sites for more resources. Excellent. John, any final thoughts?

John Doyle: 50:28

Maybe we end on an uplifting note. We're better together. So this whole field, I feel like, is predicated on a bunch of smart individuals who want to do the right thing, who want to impose costs on the adversaries. So the more opportunity we have to work with each other and grow from one another, the better off I think we're going to be. We're going to kind of grow and evolve the industry. We're going to help advance the tradecraft of practices. We're going to help... promulgate the value-add proposition to C-suite executives and others. I have been a recipient of mentorship. I have helped mentor people before. We all kind of mentor each other in a lot of ways, and that just helps us grow collectively. So just be good to one another and just try and pay it forward as best we can. That's, I guess, my uplifting way to end this recording on a Friday.

Pedro Kertzman: 51:24

Man, honestly, that resonates so much to me. I really love that. Probably that's the most interesting part. One of the most interesting parts of having this podcast is to see how the community can come together like random strangers just to help each other share knowledge with the broader CTI or cybersecurity community about the advantages of having a CTI program. And that's just so amazing. I love this attitude. And I think when you pick this specific quote unquote topic, As a final thought, I think just goes to show that it's really something important in the community. John, thank you so very much for coming to the show. I really appreciate all the insights and I really hope I'll see you around.

John Doyle: 52:18

Yeah, absolutely. Thank you again for having me. This was a ton of fun.

Rachael Tyrell: 52:23

And that's a wrap. Thanks for tuning in. If you found this episode valuable, don't forget to subscribe, share, and leave a review. Got thoughts or questions? Connect with us on our LinkedIn We'd love to hear from you. If you know anyone with CTI expertise that would like to be interviewed in the show, just let us know. Until next time, stay sharp and stay secure.