Anura PR Team
When evaluating tools that are most effective for detecting bots on your website, the best tools will have signal diversity, adaptability against sophisticated bots, accuracy against false positives, operational fit, control and practical consideration like cost. We will explore each of these in more detail below to give you an effective framework to make the best decision for your organization.
Signal Diversity
Signal diversity really means that the tool has multiple layers of detection since fraud determinations usually don’t come from a single characteristic or behavior, it comes from a combination of signals that paint a picture of whether a visitor is human or not. While every tool has it’s own algorithm to make these determinations, there are some common concepts that they should have incorporated into their process.
Environmental Analysis – Environmental analysis collects information form the visitor’s environment for things like geolocation, does the visitor have dev tools on (which is a great indicator of malicious activity but not a guarantee), IP address and more. Environmental signals are an important bot detection layer because it tells you more than other signals and can be harder to spoof. It also serves as a good foundation to set up other signals like fingerprinting.
Fingerprinting – Fingerprinting is the process of condensing a lot of information into a unique string of data that allows detection tools to keep track of that individual visitor across their journey. Fingerprinting alone won’t tell you if a visitor is a bot or not, but it is vital for tools because it is used as an identifier to attach other signals to that will make that determination.
Challenge/Response – Challenge response can also be known as “verification challenges”, which can be lines of code hidden on a page that send a query to a visitor and wait for a response (the type of response that a visitor returns could flag malicious activity if it differs from the expected response) or a simple picture grid that asks, “which of these images have cars?”. Challenge/response is a very common way to see if a visitor is acting as expected or trips any red flags to be analyzed in conjunction with other signals.
Adaptability Against Sophisticated Bots
General invalid traffic, like SEO and accessibility crawlers are easy to detect because they either announce themselves or have very common and known signals. What you really need to protect against are sophisticated bots that either hid themselves or are used in ways that mimic real humans.
AI Driven Bots - AI enabled bots are scripts written and edited by AI to perform tasks that require human interaction. Scripts used to be written manually, which made their patterns and behavior easier to detect. With the advent of AI, bots are being written and are adapting in real time to bot detection strategy. These bots are the hardest to identify, especially when relying on behavioral analysis or captcha challenges alone. (Anura does not use behavioral analysis because of its weaknesses).
Headless Browsers – Tools like puppeteer or chromium are browsers that parse website code and handle cookies like a normal browser but don’t load the visuals of a website. These tools also browse by code prompts instead of clicks, which are great for automated testing and web scraping, but is also highly associated with bots and malicious activity.
Residential Proxies - Residential Proxies are proxy servers that route internet traffic through legitimate IP addresses, usually through malware if happening for malicious purpose. There was a real world example of this on Samsung TV’s. Samsung had an app on their marketplace that was loaded with malicious malware. When a Smart TV owner downloaded the app, they unknowingly downloaded malware that allowed fraudsters and bots to use the owners IP address and devices as their environmental data (see below), making it look like they were originating from legitimate IP’s instead of data centers or bot farms. Residential proxies can make bot identification much harder, which is why you need a wide array of data points for accurate bot detection.
Accuracy and False Positives
Any tool that you consider should be focused on accuracy, but more importantly accuracy around false positives. Accurate bot detection is never a “100%” game, because bots and the people behind them are constantly evolving and changing, making complete detection impossible (this should also be a point of scrutiny for you, so if a vendor is promising 100% accurate detection they are probably lying). So, if you can’t focus on the detection accuracy, you need to focus on false positives. Letting legitimate visitors through while blocking the illegitimate ones is essential because legitimate traffic is the goal. Any tool you’re looking at should have a strong focus on low false positive rates to ensure you don’t take a hit on conversion and ROI.
How do you pick?
Understanding which characteristics of a bot detection tool is crucial to finding the best fit for your organization. Ultimately, the best tool analyzes your traffic in the most effective way that compliments your business and revenue models. Real-time analysis is vital for a CPC model, while an e-commerce store may require a higher “no false positive” accuracy score. Regardless of your model, the first step always starts with understanding your traffic quality, which most tools will analyze for free to give you an idea of the scope of your bot problem before closing a deal. Start with Anura’s free traffic quality audit to better understand your traffic today.
