Bot Detection 101: What Every Marketer Should Know

For any organization running any kind of online advertising campaign, invalid traffic (IVT) from bots is an enormous issue. To put a dent in the ad fraud caused by bots, bot detection is vital. After all, how can you put a stop to bots if you can’t detect them in the first place?

Unfortunately, many organizations struggle with detecting bot traffic and the solutions they use aren’t all created equally. Before attempting to stem the unending tide of bots that may target your online marketing efforts, it’s important to learn more about bots, how they work, and how to detect them.

What Is a Bot?

The term “bot” refers to an automated software program designed to carry out a specific task. Estimates vary, but sources like CPO Magazine state that “bot traffic made up 42.3% of all internet activity in 2021.” Bots can be divided into two categories: “Good” bots and “Bad” bots. What separates the two?

Good Bot

A “good” bot is used for benign tasks. For example, a web browser might save a user’s address information when they fill out an online form and, the next time the user is asked for their address, the bot will fill it out in an instant rather than making the user type it out manually.

Bad Bot

A “bad” bot may do something similar, but be used for more nefarious purposes. For example, a bad bot might check an entire table of real consumers’ address information and use it to fill out online forms—posing as a unique website visitor each time.

Here, what separates the good from the bad is what the bot is used for. Like virtually any tool, a bot can be used to help people or to cause harm—it’s all up to the person who programs the bot and the one using the bot.

What Is Invalid Traffic and How Does It Relate to Bots?

Bots are generally classified as “invalid traffic” by marketers, since the bots are not valid targets for converting from leads into customers.

Invalid traffic can be further divided into two categories based on their level of sophistication:

• General invalid traffic (GIVT) consists of the simplest bots that are the easiest to detect. A lot of “good” bot traffic is GIVT since they aren’t meant to be attempting to fool a target’s bot detection solutions. However, fraudsters also employ a lot of GIVT because simpler bots are easier to make and they may still work against some of their targets.
• Sophisticated invalid traffic (SIVT) consists of bots that are more capable and are often designed to thwart a target’s cybersecurity and fraud prevention tools. For example, SIVT bots might imitate how a human would use a website to disguise the fact that they’re bots. SIVT is extremely common in ad fraud schemes.

What About Botnets?

A botnet is a massive collection of bots—typically installed on devices infected with zombie bot malware that allows the bot controller to remotely hijack some of each device’s processing power to run the bot program. These botnets can be used for all kinds of malicious activity—such as conducting ad fraud, running DDoS attacks, or helping to grow the botnet larger by infecting any networks and devices connected to the zombie bot-infected device.

Types of Bots and Their Impacts

There are many varieties of bots used for modern ad fraud schemes. This can make bot detection a bit challenging since the warning signs of bot traffic might vary from one type of bot to the next. To help you get a grasp of the bot-based threats you may face, here’s a list of the different kinds of bots, what they’re used for, and some early indications of bot traffic:

1. Form Bots

These are bots that are specifically designed to fill out online forms on the behalf of their controller. The programmer designs the bots to recognize form fields and fill them out automatically to provide appropriate responses to common questions.

Form bots can be further subdivided into distinct categories based on how they’re used.

Two examples of form bots include:

• Lead generation fraud form bots that are often used in lead gen fraud schemes. Here, the bots are directed to fill out a target’s lead generation forms online while giving credit for the new “lead” to the fraudster. In many cases, these form bots use tables of real consumer data (often stolen or resold without consent) to populate the lead gen forms. Because the data is real, it often looks more like a lead is legitimate.
• Survey bots that specifically target surveys and polls to skew the results data. Aside from what they target, they operate nearly identically to lead gen fraud bots. The major impact isn’t direct financial losses—instead, these bots hurt an organization’s ability to make data-driven decisions. Survey bots frequently target market research groups, businesses, and political survey efforts.

2. Spam Bots

Spam bots are bots used to repeatedly post the same messages repeatedly on different websites and social media platforms. These kinds of bots may be used against a business by spreading negative reviews and fake stories about a company that, eventually, real humans might pick up and start sharing.

These bots can have devastating long-term impacts on a business’ reputation if allowed to post their lies without being challenged.

3. Social Media Account Bots

Social media bots are programmed to automatically manage social media accounts on the behalf of someone else. These bots can be programmed to set up whole new accounts or to manage already-existing accounts on behalf of a scammer.

Some applications for social media bots include:

• Artificially inflating an influencer’s subscriber/follower count (often used in affiliate fraud to make the fraudster look like an attractive partner).
• Sabotaging a social media account by providing low-quality engagement.
• Uploading spam comments with malicious links under other posts—overlapping somewhat with spam bots.

4. Backlink Bots

In online marketing circles, backlinks (links to your website from another site) are often considered incredibly important. A large number of quality backlinks tells Google and other search engines that you have a quality website that people want to engage with—boosting your domain authority and thus your position in search engine results.

However, there is such a thing as a bad backlink. When lots of spammy websites link to yours, Google and other search engines may actually punish your ranking in their search engine results—pushing you lower on the list where people aren’t as likely to see your website. Bot programs can be used to automate this process—the fraudster simply adds your website URLs to the content table the bot pulls from and the bot puts it on as many bad websites as it can.

Some cybercrooks try to take advantage of this fact to blackmail companies or to hurt their competitors by using bots to add bad backlinks to spammy, low-quality websites. In the case of crooks looking to make a quick buck, they’ll often threaten something along the lines of “give me money or else I’ll tank your website so nobody sees it.” In the case of underhanded competitors, they’ll post the backlinks without you being any the wiser if they can.

This is why it’s important to run a check of the backlinks going to your website and use Google’s disavow tool to get the search engine to ignore spam links.

5. Impression Bots/Page Refresher Bots

If you’re running an ad campaign where you’re paying based on the number of times someone “sees” and ad (i.e., the number of impressions it generates), then you may soon find your ads being targeted by impression or page-refreshing bots.

These bots continuously load web pages with your ads to generate as many impressions as possible in as short a time as possible—forcing you to pay for completely invalid traffic to enrich the fraudster behind the bot. More sophisticated impression bots may use device spoofing techniques in between page refreshes to give the illusion that a variety of sources are viewing your ads. Simpler refresh bots just refresh the page as fast as possible.

6. Click Bots

Click bots go a step beyond impression bots—generating a “click” on an ad to earn a fraudster money from your pay-per-click ad campaigns. These bots don’t have to be very sophisticated to work—though some fraudsters do combine them with device spoofing techniques to try to make their fraud somewhat less obvious.

By clicking on ads repeatedly and claiming credit, fraudsters can quickly drain a click-based ad campaign of money while preventing the victim from generating any real results.

Bot Detection Basics

So, how can you know if bots are targeting your ad campaigns or being used to damage your business in other ways? Here are a few things you can do:

Keep an Eye Out for the Bot Activity Warning Signs

Malicious bot activity often results in a few oddities with your website traffic or your marketing activity results.

Some of the warning signs of bot activity include:

• A sharp increase in impressions and clicks without any increase in leads generated. This could be an indication that impression bots and click bots are targeting your ad campaigns.
• An influx of comments on your website blog posts, social media posts, or online videos with spammy links or suspiciously similar complaints (ones where the only difference is the name of the commentor) can be a sign that your business is being targeted by spam bots.
• A major drop in website traffic without any changes either on your part or to the Google algorithm, which does change from time to time. This could be an indication of someone using bad backlinks to sabotage your website’s search engine ranking.
• A sudden increase in complaints from leads about not opting into receiving communications from your organization could be a clue that form bots are being used to generate fake leads using real consumers’ information.
• A marketing affiliate not generating results proportional to the apparent size of their audience could be an indication that they used social media bots to artificially inflate their subscriber/follower numbers (or that their audience just wasn’t a good fit for your organization).
• Survey results that fall well outside of expectations or where the majority of responses all seem to be nearly identical. This could indicate that a survey bot is targeting your surveys and tweaking the responses to invalidate your survey results.

Use Honeypot Form Fields

While not guaranteed to thwart all form bots—especially more sophisticated ones programmed to work only on the specific responses visible to humans—honeypot form fields can be a valuable tool for stopping many form bots.

Honeypot form fields only exist in the webpage’s code and aren’t visible to human users. However, bots don’t “see” what’s on the screen of the page the way that humans do. Instead, they read the code of the page to determine what’s there. So, form bots may try to fill out the form fields that exist only in the hidden page code.

When you see form submissions that include responses to the hidden form fields, you’ll know that the submission is a fake and that you should ignore that “lead.” If you get a lot of these invalid leads and they’re all attributable to one affiliate marketing partner, that’s a strong indication that you should cut that person out of your affiliate program.

Periodically Check the National Do-Not-Call Registry

The National Do-Not-Call (DNC) Registry is a list of people who have specifically registered their desire to not receive unwanted marketing materials or messages from businesses. If you have numerous leads who are on the DNC list, it could be a sign that someone is using bots to fill your lead generation forms with stolen information.

This is why it’s important to check the DNC list periodically to verify your leads. In addition to rooting out potential lead gen fraud, inspecting the DNC Registry is important for maintaining TCPA compliance since it goes to demonstrate the organization’s commitment to preventing unwanted communications.

Use Verification Emails to Confirm New Leads

Verifying that communications are wanted is an important step in both avoiding complaints and in spotting bots. When adding new leads, sending out a verification email with a link can help you ensure that these leads are from actual people and not just bots posing as people.

While some sophisticated fraudsters might set up fake email addresses that they control (and thus, can click on the verification email), others may use real email addresses from actual consumers to make their fake leads look authentic. When the real owner of the email address gets the verification email, they may not even notice it (since they won’t be on the lookout for it) or they may ignore it. In either case, the link won’t get clicked and you can avoid adding the fake lead.

This added verification does have pros and cons. For example, a genuine lead may not want to jump through the added hoop of going to their email inbox and looking for your verification email. So, the added friction can lead to losing genuine leads. On the other hand, it can be useful for demonstrating TCPA compliance since you’ll have that electronic signature showing that there was a clear opt-in from the marketing contact to receive promotional materials.

Use an Ad Fraud Solution

Ad fraud solutions can be your best tool for putting a stop to ad fraud and for detecting bot activity in your online marketing efforts. With the right solution, you can detect ad fraud and bots as they hit your website—putting a stop to the fraud before it has a chance to harm your marketing budget or your company.

However, not all ad fraud solutions are the same. Some make grandiose claims to be certified by organizations such as the Trustworthy Accountability Group (TAG), but aren’t actually certified at all. Others may claim to use one airtight metric to detect ad fraud 100% accurately, but bot fraud—especially the most sophisticated invalid traffic from modern bots—is too complex to be reliably identified using any single metric.

It's important to verify an ad fraud solution provider’s claims before paying for their solution.

Some ways to do that include:

• Checking the website of any certification group the solution provider claims to be certified by. For example, Anura is TAG certified against fraud to detect fraudulent activity in real time and can be found on the TAG registry.
• Using a free trial of the solution in question. As the saying goes, “the proof is in the pudding.” Trying a solution out before committing to buying it can do a lot to tell you about how effective the solution is. Not offering any hands-on time with a product before you sign a contract can be a powerful indication (or condemnation) of an ad fraud solution’s quality.
• Checking out how the ad fraud solution worked out for its other users. Look at case studies of the product and try to reach out to their other customers for comments about how well the ad fraud solution worked.

Need help finding bot traffic and identifying fake leads in your ad campaigns? Reach out to Anura now to start protecting your company from bot-based ad fraud!