Sitting in the first Keynote for the 2013 Techno Security and Forensics Investigation Conference, I was not surprised to hear Kevin Mandia say that in their recent investigations, they had found anti-virus installed and working with the latest definitions. Yet these systems were still infected with malware. In short, AV had failed to stop the malware.

So why is there a corporate and personal insistence on having anti-virus installed, especially in the Windows world? It's probably because we value its protection too heavily. To be blunt, AV stops the easy attacks. It typically stops the attacks that we've known about, attacks that will get you if you have no protection at all. So while AV has some worth, it's not the magical full plate armor that too many folks think it is. Why is that?

It's Mostly Signature Based

It used to be that we saw AV definitions update about once a month. Then that shortened to about every two weeks. Then it was weekly. Then it was even daily. However, nowadays a lot of organizations are pulling AV definitions down more often than once a day. I know some organizations have hourly checks to pull the latest definitions and get them distributed to their AV clients. Why are so many updates required? Quite simply, because AV is still primarily signature based. What that means is that the AV companies are effectively "fingerprinting" the malware. Those "fingerprints" are what are used to detect the malware. Those are what we mean by AV definitions. Therefore, as you discover new malware, you have to develop new definitions because you're going to have to include their "fingerprints." And every once in a while there are false positives.

This alone should cause you to pause. What if the malware isn't known about, will there be definitions? Obviously, there won't be. Therefore, if we're speaking of brand new or very tightly targeted attacks, AV won't detect the malware based on signatures. Therefore, the malware gets in and it runs, most of the time. That's why no AV gets a 100% rate.

What about Behavior Based Detection?

This is a great idea, in theory. The problem on a live system is that what an AV application may detect as potential malicious behavior is, in fact, legitimate. For instance, a lot of malware creates an HTTP connection to either grab updates or grab additional malware to bring down. To the OS, this doesn't look very different than when you click on that link in your favorite Twitter client. As a result, AV has to be careful not to be too aggressive. Because of this, there's a lot of latitude for malware to operate. And if you don't think malware writers are aware of behavior heuristics in AV programs, think again.

Malware Gets Tested, and Tested Well

Unlike some code, malware gets tested before "production." We're past the stage of folks writing viruses that are just comical in nature. Now we're talking about stealing money or stealing secrets. That means you've got professional players in the game. When you have professionals, and when you talk about the worth of the data or assets they are stealing, they want to make sure their stuff works. When it comes to folks on the criminal financial side, we're talking millions of dollars on the line. Therefore, they test.

How do they test? Basically, they run their malware against the known AV engines and see if the engines get detected. At the very least, something like VirusTotal gets used. Therefore, the attackers already know if their malware is going to get detected. If it does, they work on it. By the way, when they see definitions pop up for one of their malware tools, they know to deploy a new tool. In this regard, AV actually works against those trying to do the investigations as to who is hacking and what they're up to.

Not Everything Makes a Definition

Let's say a particular piece of malware gets submitted. Does that mean it automatically gets included in the AV definitions? The short answer is, "No." Definitions are determined by a whole host of factors. Among them is if it's clearly identified as a threat. Why do I say that? Well, take Stuxnet, for instance. It was discovered in 2010 but believed to have been created in 2009. Only that's not the whole story. It seems that Symantec found traces of it way back in 2005.

Another thing to remember is the sheer volume of malware submissions in today's world. Before, researchers had time to hand check every submission. It was still practical to do so. Nowadays submissions run through a filter of automated checks. Why is that? I have seen estimates in the neighborhood of 250,000 submissions a day. I know in 2009 the number was about 50,000 per day. Now, some of those are already known. Others aren't malware. So it's very possible something with very new behavior, especially if it doesn't appear to be aggressively malicious, can be missed.

As to what other factors there are? Part of it is impact and spread. We know this because when we've had major outbreaks, we've seen AV companies rush to get signatures out the door. With targeted attacks, there aren't many samples and there aren't many infections. It may be harder to get a signature together to put in a definition. And that means the malware stays viable all the longer.

So Should We Trash Anti-Virus?

No, it still serves a purpose. Just realize its limited effectiveness. It's going to stop most known and common threats. It's not going to stop a targeted threat. It's not going to stop a threat from what we're calling Advanced Persistent Threats (APTs). It's not going to stop a brand new threat that the AV companies haven't had time to analyze. Therefore, AV is only a small part of the overall security picture. Too many organizations and people rely on it as a major player in a security response. It shouldn't be. Those days are gone.