Herding Cattle: ThreatConnect’s Vision for Better Intel Feeds

Introducing CAL 2.4

We’re thrilled to announce the release of ThreatConnect’s CAL™ (Collective Analytics Layer) 2.4.  As always, our releases strive to bring you more data, better data, and actionable insights. In that spirit, we’ve made a massive commitment to expanding CAL’s ability to deliver those actionable insights with our newest featureset, CAL Feeds. I’m going to call them CALFs in this article, because I think they’re more than just feeds and I like the idea of writing an article full of cow puns.

The concept of a feed isn’t a new one – at a tactical level, we’re all familiar with ingesting them and incorporating them into our defense. At a strategic level, CAL has been an innovator in the way that we measure feed performance in customer ecosystems via Report Cards. I think CALFs deserve their own noise because they’re trying to steer us into finding pockets of intelligence that haven’t blown up yet. These are interesting areas of the internet where we can not only glean intel, but sharpen the saw and get better at gleaning intel.

My Beef with the Status Quo

We built CAL for two reasons: we knew that there were some questions that required specialized technology to solve, and we knew that ultimately security practitioners could benefit from an aperture into “we” instead of focusing on “me.”

The methodology for you to use CAL in your day-to-day job reflected that. Starting out, CAL would begin to pepper in various insights about how “we” see a threat. How broadly targeted is this activity? Who owns this infrastructure? We started to add in additional insights – reputation scores, indicator status, classifications of indicators. There’s a lot of value here – we can help your team improve their accuracy and efficiency as they go through investigations or day-to-day research. Still, all of these insights had one one limitation: CAL can tell you about an indicator, if you know to ask about it.

Greener Pastures

That constraint always bothered me as a limiting factor of CAL. It limits your ability to move proactively, at times limiting you to only improve your reaction time. CAL could tell you that the IP address in question is bad, or that a host was recently unparked. Those are helpful insights once that intelligence is in your crosshairs, but there’s a larger problem we wanted to take a bite out of. How do you get the right things in your crosshairs? This presents a big hurdle when you’re trying to align your security goals and your business goals: deciding what you care about, and testing that hypothesis, requires a bigger aperture.

This is where we’ve taken strides in finding new ways to deliver intelligence that we think is interesting. We’re not making promises that we’ve developed some whiz-bang algorithm to detect all malicious activity on the internet. We’re not appending a random number generator after the letters “APT” to start labeling data. We’ve married the immense CAL dataset and analytics with the tradecraft of our Research team to identify some pockets of intelligence that are fertile hunting grounds. By packaging this intelligence and pushing it directly to your ThreatConnect instance, we’re allowing you to start to get back onto proactive footing so that you can be more predator than prey.

Let’s flex those CALFs

As with most projects like this, words are cheap and charts are cool. Let’s take a look at our initial lineup of the four CAL Feeds (affectionately known as CALFs) that are available to you:

  1. CAL Suspicious New Resolution IPs – We are ingesting nearly 100 open source feeds into CAL to improve our understanding of intel. So when we see something new, it’s at least interesting. This feed of IPs are the DNS resolutions from identified malicious hosts that aren’t being reported anywhere else. Here’s your chance to shift to the proactive – don’t wait for an analyst to ingest, triage, document, and share these IPs in their “spare time.” Let our code do it for you, as these malicious hosts come online you’ll have the first peek at their underlying infrastructure.
  2. CAL Suspicious Newly Registered Domains – I’ve always been intrigued by newly registered domains (abbreviated as NRDs, or I call them “nerds” when I want to tell someone else how cool they are in real life). NRD’s aren’t inherently malicious – new stuff gets registered every day! But some subset of it is at least suspicious or interesting. By virtue of being new, they eliminate one of the big questions we always have to ask with intel: “how old is it?” We’ve identified NRDs that we think are leveraging suspicious infrastructure, which narrows down the pool. These domains can be a rich hunting ground for reusing infrastructure, registration techniques, and more!
  3. CAL Suspicious Nameservers – We’ve paired CAL’s dataset with Kyle Ehmke’s analysis to figure out a way to identify suspicious nameservers at scale. Nameserver usage (and reusage) can help us identify shady neighborhoods on the internet, and thus adds an important datapoint to any hosts that use them.This has been such a helpful watchlist for us to define and use internally – when paired with something like DomainTools Iris we’re able to keep our “police blotter” up to date so we can hunt faster.
  4. CAL Suspected Ranking Manipulators – When we started to explore NRDs, we noticed an interesting trend. There are a number of really new domains that seem to be highly ranked in some of the industry’s “Top 1 Million Websites” list. There has been a plethora of research on the ability to manipulate rankings (I would highly recommend looking at Tranco’s thoughts on the subject). So that naturally led us to ask: who’s doing the manipulation here? These domains may not be directly targeted against you, but if you’re a research geek like us you’re probably just as curious. These domains have a “weirdness” to them that we’re hoping to sink our teeth into. That weirdness smacks of nefarious activity that we don’t know much about and may not want to leave unchecked.

Strap on the Feed bag

If you’re excited to get started with some of the cool data that CAL Feeds can provide, it’s easy to get started. Just like our supported open source feeds, CALFs are available to system administrators through our TC Exchange Feeds Catalog. And just like the other feeds, they’ll get a report card and can be enabled with the click of a button:

Upon enabling a CAL Feed, its Source will be automatically created and configured. It will start populating automatically, with a predefined window of historical data being created (and aged out) appropriately with ratings and tags and everything!

Please send us a tweet (@ThreatConnect) and let us know what you think of the new CAL Feeds! We’re always looking to improve them, and we’ll be adding more in the future as we find opportunities to take novel looks at the massive dataset that’s swirling around us!




Drew Gidwani
About the Author
Drew Gidwani

Drew Gidwani is the Director of Analytics at ThreatConnect. He drives the data modeling, collection, and analytics both within the core ThreatConnect platform and in CAL. Previously, Drew worked for the Department of Defense where he leveraged his varied analysis experiences to scale growing intelligence teams in the face of the ever-changing threats we face today. Drew holds a B.S. from Carnegie Mellon University and an M.S. from Johns Hopkins University. He currently resides in Maryland with his fierce warrior dog named Gimli.