{"id":7902,"date":"2025-11-24T10:18:01","date_gmt":"2025-11-24T10:18:01","guid":{"rendered":"https:\/\/unitconversion.io\/blog\/?p=7902"},"modified":"2025-11-24T10:27:40","modified_gmt":"2025-11-24T10:27:40","slug":"how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss","status":"publish","type":"post","link":"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/","title":{"rendered":"How BulletProof Security\u2019s Auto Block Banned Search Engine Crawlers and the Exception Rules That Fixed Indexing Loss"},"content":{"rendered":"<p>Website security is a top priority for any site owner, particularly for those running WordPress installations. Among the many tools available to safeguard websites, <strong>BulletProof Security<\/strong> has proven to be a powerful all-in-one solution for securing WordPress sites. However, even the most robust security tools come with trade-offs. One of BulletProof Security\u2019s features\u2014auto-blocking suspicious crawlers\u2014led to an unexpected side effect: legitimate search engine bots were being banned, causing critical indexing losses. Fortunately, implementing <em>exception rules<\/em> helped resolve the issue and restore search visibility.<\/p>\n<h2>TL;DR:<\/h2>\n<p>BulletProof Security, a popular WordPress security plugin, can automatically block bots and crawlers that appear suspicious. Unfortunately, the plugin\u2019s auto-block feature sometimes bans legitimate search engine bots like Googlebot and Bingbot, leading to lost indexing. Website owners noticed a drop in organic search presence before identifying the issue. Creating precise exception rules allowed approved search engine crawlers to bypass security filters while continuing to block malicious traffic effectively.<\/p>\n<h2>The Problem: Auto Block Feature Interferes with Crawlers<\/h2>\n<p>BulletProof Security includes a feature known as <strong>Auto Blocking<\/strong> that prevents potential threats from accessing the site. This feature works by detecting request patterns in headers, user-agents, and IP behaviors that resemble known hacking attempts or unwanted bots. While beneficial in many cases, it also means that some better-known bots, including those from search engines, can be mistaken for threats under certain server configurations or misidentifications in user-agent strings.<\/p>\n<p>For webmasters relying on organic traffic to drive visibility and revenue, even a temporary loss in indexing can lead to:<\/p>\n<ul>\n<li><strong>Decreased search engine rankings<\/strong><\/li>\n<li><strong>Lower organic traffic<\/strong><\/li>\n<li><strong>Impaired SEO performance<\/strong><\/li>\n<li><strong>Delayed content discovery<\/strong><\/li>\n<\/ul>\n<p>Once these legitimate bots are blocked, they can no longer crawl and index the site properly, causing substantial issues in discoverability and rankings.<\/p>\n<img loading=\"lazy\" decoding=\"async\" width=\"1200\" height=\"675\" src=\"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2023\/05\/top-5-security-plugins-to-keep-your-wordpress-site-safe.jpg\" class=\"attachment-full size-full\" alt=\"\" \/>\n<h2>What Are Crawlers and Why Are They Blocked?<\/h2>\n<p>Search engine crawlers\u2014like Googlebot, Bingbot, and YandexBot\u2014regularly visit websites to scan and index them for search engine results. To security plugins like BulletProof Security, these bots can sometimes mimic the behavior of malicious scripts or attackers, especially if:<\/p>\n<ul>\n<li>Their user-agent strings are misused by non-legitimate bots attempting to pass off as trusted crawlers<\/li>\n<li>They make high-frequency requests that resemble a brute-force or DDoS attack<\/li>\n<li>They access directories or files that trigger security rules<\/li>\n<\/ul>\n<p>When bulletproof in nature, a firewall will act on any suspicious pattern. While that&#8217;s often what is expected, the inability to differentiate between real and spoofed bots can lead to false positives\u2014resulting in blocked search engine traffic.<\/p>\n<h2>Identifying the Issue: Signs of Search Engine Blocking<\/h2>\n<p>Most users only become aware of this issue after experiencing one or more of the following:<\/p>\n<ul>\n<li><em>Google Search Console reports a sudden increase in crawl errors<\/em><\/li>\n<li><em>Pages are no longer showing up in Google search results<\/em><\/li>\n<li><em>A significant drop in daily organic traffic<\/em><\/li>\n<li><em>Server logs showing 403 errors for known bots<\/em><\/li>\n<\/ul>\n<p>Upon deeper investigation, webmasters searching server logs may uncover that user-agents attributed to Googlebot and Bingbot are receiving HTTP 403 or 503 responses \u2014 classic signs that the site\u2019s firewall or security plugin is blocking them.<\/p>\n<h2>Temporary Fix Attempts That Didn\u2019t Work<\/h2>\n<p>In a rush to reverse traffic losses, website owners may attempt several workarounds:<\/p>\n<ul>\n<li>Disabling security plugins entirely<\/li>\n<li>Whitelisting entire IP ranges<\/li>\n<li>Turning off all request filtering rules in .htaccess<\/li>\n<\/ul>\n<p>However, these actions often create temporary exposure to real vulnerabilities without solving the issue at its core. The goal should always be to protect the site <em>without<\/em> blocking legitimate traffic\u2014a much more nuanced balancing act.<\/p>\n<h2>The Real Solution: Exception Rules for Search Engine Bots<\/h2>\n<p>The final resolution came in the form of <strong>exception rules<\/strong> within BulletProof Security\u2019s configuration. These are custom rules that allow certain user-agents or IP addresses to bypass standard filtering mechanisms, making it possible to keep protection levels high while ensuring good bots are given a pass.<\/p>\n<p>The most effective way to implement exception rules involves:<\/p>\n<ol>\n<li>Identifying legitimate search engine IP ranges (using tools like <a href=\"https:\/\/developers.google.com\/search\/docs\/advanced\/crawling\/verifying-googlebot\" target=\"_blank\" rel=\"noopener\">Google\u2019s official IP verification<\/a>)<\/li>\n<li>Whitelisting verified user-agents (e.g., &#8220;Mozilla\/5.0 (compatible; Googlebot\/2.1; +http:\/\/www.google.com\/bot.html)&#8221;)<\/li>\n<li>Creating hard-coded rules in .htaccess or using BPS&#8217; built-in features to allow these bots while still filtering unknown and malicious agents<\/li>\n<\/ol>\n<img loading=\"lazy\" decoding=\"async\" width=\"1080\" height=\"720\" src=\"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2025\/11\/white-and-red-do-not-enter-signage-firewall-rules-exception-list-googlebot-access.jpg\" class=\"attachment-full size-full\" alt=\"\" srcset=\"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2025\/11\/white-and-red-do-not-enter-signage-firewall-rules-exception-list-googlebot-access.jpg 1080w, https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2025\/11\/white-and-red-do-not-enter-signage-firewall-rules-exception-list-googlebot-access-300x200.jpg 300w, https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2025\/11\/white-and-red-do-not-enter-signage-firewall-rules-exception-list-googlebot-access-1024x683.jpg 1024w, https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2025\/11\/white-and-red-do-not-enter-signage-firewall-rules-exception-list-googlebot-access-768x512.jpg 768w\" sizes=\"(max-width: 1080px) 100vw, 1080px\" \/>\n<h2>Best Practices for Implementing Exception Rules<\/h2>\n<p>To avoid running into the same issue again, webmasters should apply the following best practices when configuring security tools:<\/p>\n<ul>\n<li><strong>Use BPS\u2019 Logging Capabilities:<\/strong> Review logs frequently to monitor which user-agents and IPs are getting blocked.<\/li>\n<li><strong>Verify Crawler Authenticity:<\/strong> Not all crawlers that identify as Googlebot are genuine. Use DNS reverse lookups to confirm authenticity.<\/li>\n<li><strong>Use Specific Rules Rather Than Global Opens:<\/strong> When allowing access, avoid opening entire IP ranges unless they are from a verified source.<\/li>\n<li><strong>Test After Each Change:<\/strong> Use tools like the Fetch as Google feature in Search Console to verify whether Google can now access your site again.<\/li>\n<\/ul>\n<h2>The Result: Full Indexing Recovery<\/h2>\n<p>Once exception rules were correctly implemented, legitimate bots were once again able to crawl and index the website. Search traffic returned to expected levels within a couple of weeks as pages were re-discovered and rankings were reinstated.<\/p>\n<p>This experience served as a vital reminder that web security and SEO are intricately linked. Even though security cannot be compromised, neither should it unintentionally impede visibility. Configuring tools like BulletProof Security to distinguish between threats and valuable crawlers ensures that your website remains safe <em>and<\/em> seen.<\/p>\n<h2>Conclusion<\/h2>\n<p>BulletProof Security remains a highly effective plugin for protecting WordPress sites. However, its automated blocking features can sometimes be too strict\u2014mistaking helpful bots for hostile agents. The key takeaway is that thoughtful configuration, especially the use of <strong>exception rules<\/strong>, is necessary to maintain a healthy balance between robust security and strong search engine visibility.<\/p>\n<p>For website owners and developers, the lesson is simple but clear: trust your tools, but always verify their behavior. Use logging, IP verification, and exception protocols to ensure that BulletProof Security works for your SEO\u2014<em>not against it<\/em>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Website security is a top priority for any site owner, particularly for those running WordPress installations. Among the many tools available to safeguard websites, <strong>BulletProof Security<\/strong> has proven to be a powerful all-in-one solution for securing WordPress sites. However, even the most robust security tools come with trade-offs. One of BulletProof Security\u2019s features\u2014auto-blocking suspicious crawlers\u2014led to an unexpected side effect: legitimate search engine bots were being banned, causing critical indexing losses. Fortunately, implementing <em>exception rules<\/em> helped resolve the issue and restore search visibility. <a href=\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/\" class=\"read-more\">Read more<\/a><\/p>\n","protected":false},"author":79,"featured_media":3745,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[665],"tags":[],"class_list":["post-7902","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50","no-featured-image-padding"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>How BulletProof Security\u2019s Auto Block Banned Search Engine Crawlers and the Exception Rules That Fixed Indexing Loss - Unit Conversion Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How BulletProof Security\u2019s Auto Block Banned Search Engine Crawlers and the Exception Rules That Fixed Indexing Loss - Unit Conversion Blog\" \/>\n<meta property=\"og:description\" content=\"Website security is a top priority for any site owner, particularly for those running WordPress installations. Among the many tools available to safeguard websites, BulletProof Security has proven to be a powerful all-in-one solution for securing WordPress sites. However, even the most robust security tools come with trade-offs. One of BulletProof Security\u2019s features\u2014auto-blocking suspicious crawlers\u2014led to an unexpected side effect: legitimate search engine bots were being banned, causing critical indexing losses. Fortunately, implementing exception rules helped resolve the issue and restore search visibility. Read more\" \/>\n<meta property=\"og:url\" content=\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/\" \/>\n<meta property=\"og:site_name\" content=\"Unit Conversion Blog\" \/>\n<meta property=\"article:published_time\" content=\"2025-11-24T10:18:01+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-11-24T10:27:40+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2023\/05\/top-5-security-plugins-to-keep-your-wordpress-site-safe-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"675\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Olivia Brown\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Olivia Brown\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/\"},\"author\":{\"name\":\"Olivia Brown\",\"@id\":\"https:\/\/unitconversion.io\/blog\/#\/schema\/person\/4ea06b340c4660f4a04bd6d58c582b69\"},\"headline\":\"How BulletProof Security\u2019s Auto Block Banned Search Engine Crawlers and the Exception Rules That Fixed Indexing Loss\",\"datePublished\":\"2025-11-24T10:18:01+00:00\",\"dateModified\":\"2025-11-24T10:27:40+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/\"},\"wordCount\":1013,\"publisher\":{\"@id\":\"https:\/\/unitconversion.io\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2023\/05\/top-5-security-plugins-to-keep-your-wordpress-site-safe-1.jpg\",\"articleSection\":[\"Blog\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/\",\"url\":\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/\",\"name\":\"How BulletProof Security\u2019s Auto Block Banned Search Engine Crawlers and the Exception Rules That Fixed Indexing Loss - Unit Conversion Blog\",\"isPartOf\":{\"@id\":\"https:\/\/unitconversion.io\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2023\/05\/top-5-security-plugins-to-keep-your-wordpress-site-safe-1.jpg\",\"datePublished\":\"2025-11-24T10:18:01+00:00\",\"dateModified\":\"2025-11-24T10:27:40+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/#primaryimage\",\"url\":\"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2023\/05\/top-5-security-plugins-to-keep-your-wordpress-site-safe-1.jpg\",\"contentUrl\":\"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2023\/05\/top-5-security-plugins-to-keep-your-wordpress-site-safe-1.jpg\",\"width\":1200,\"height\":675},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/unitconversion.io\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How BulletProof Security\u2019s Auto Block Banned Search Engine Crawlers and the Exception Rules That Fixed Indexing Loss\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/unitconversion.io\/blog\/#website\",\"url\":\"https:\/\/unitconversion.io\/blog\/\",\"name\":\"Unit Conversion Blog\",\"description\":\"On conversion and other things :)\",\"publisher\":{\"@id\":\"https:\/\/unitconversion.io\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/unitconversion.io\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/unitconversion.io\/blog\/#organization\",\"name\":\"Unit Conversion Blog\",\"url\":\"https:\/\/unitconversion.io\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/unitconversion.io\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2021\/01\/uclogo.png\",\"contentUrl\":\"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2021\/01\/uclogo.png\",\"width\":500,\"height\":500,\"caption\":\"Unit Conversion Blog\"},\"image\":{\"@id\":\"https:\/\/unitconversion.io\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/unitconversion.io\/blog\/#\/schema\/person\/4ea06b340c4660f4a04bd6d58c582b69\",\"name\":\"Olivia Brown\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/unitconversion.io\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/441e8f5d29c2bd1022936f38e27eee93?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/441e8f5d29c2bd1022936f38e27eee93?s=96&d=mm&r=g\",\"caption\":\"Olivia Brown\"},\"description\":\"I'm Olivia Brown, a tech enthusiast and freelance writer. My focus is on web development and digital tools, and I enjoy making complex tech topics easier to understand.\",\"url\":\"https:\/\/unitconversion.io\/blog\/author\/olivia\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How BulletProof Security\u2019s Auto Block Banned Search Engine Crawlers and the Exception Rules That Fixed Indexing Loss - Unit Conversion Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/","og_locale":"en_US","og_type":"article","og_title":"How BulletProof Security\u2019s Auto Block Banned Search Engine Crawlers and the Exception Rules That Fixed Indexing Loss - Unit Conversion Blog","og_description":"Website security is a top priority for any site owner, particularly for those running WordPress installations. Among the many tools available to safeguard websites, BulletProof Security has proven to be a powerful all-in-one solution for securing WordPress sites. However, even the most robust security tools come with trade-offs. One of BulletProof Security\u2019s features\u2014auto-blocking suspicious crawlers\u2014led to an unexpected side effect: legitimate search engine bots were being banned, causing critical indexing losses. Fortunately, implementing exception rules helped resolve the issue and restore search visibility. Read more","og_url":"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/","og_site_name":"Unit Conversion Blog","article_published_time":"2025-11-24T10:18:01+00:00","article_modified_time":"2025-11-24T10:27:40+00:00","og_image":[{"width":1200,"height":675,"url":"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2023\/05\/top-5-security-plugins-to-keep-your-wordpress-site-safe-1.jpg","type":"image\/jpeg"}],"author":"Olivia Brown","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Olivia Brown","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/#article","isPartOf":{"@id":"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/"},"author":{"name":"Olivia Brown","@id":"https:\/\/unitconversion.io\/blog\/#\/schema\/person\/4ea06b340c4660f4a04bd6d58c582b69"},"headline":"How BulletProof Security\u2019s Auto Block Banned Search Engine Crawlers and the Exception Rules That Fixed Indexing Loss","datePublished":"2025-11-24T10:18:01+00:00","dateModified":"2025-11-24T10:27:40+00:00","mainEntityOfPage":{"@id":"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/"},"wordCount":1013,"publisher":{"@id":"https:\/\/unitconversion.io\/blog\/#organization"},"image":{"@id":"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/#primaryimage"},"thumbnailUrl":"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2023\/05\/top-5-security-plugins-to-keep-your-wordpress-site-safe-1.jpg","articleSection":["Blog"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/","url":"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/","name":"How BulletProof Security\u2019s Auto Block Banned Search Engine Crawlers and the Exception Rules That Fixed Indexing Loss - Unit Conversion Blog","isPartOf":{"@id":"https:\/\/unitconversion.io\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/#primaryimage"},"image":{"@id":"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/#primaryimage"},"thumbnailUrl":"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2023\/05\/top-5-security-plugins-to-keep-your-wordpress-site-safe-1.jpg","datePublished":"2025-11-24T10:18:01+00:00","dateModified":"2025-11-24T10:27:40+00:00","breadcrumb":{"@id":"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/#primaryimage","url":"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2023\/05\/top-5-security-plugins-to-keep-your-wordpress-site-safe-1.jpg","contentUrl":"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2023\/05\/top-5-security-plugins-to-keep-your-wordpress-site-safe-1.jpg","width":1200,"height":675},{"@type":"BreadcrumbList","@id":"https:\/\/unitconversion.io\/blog\/how-bulletproof-securitys-auto-block-banned-search-engine-crawlers-and-the-exception-rules-that-fixed-indexing-loss\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/unitconversion.io\/blog\/"},{"@type":"ListItem","position":2,"name":"How BulletProof Security\u2019s Auto Block Banned Search Engine Crawlers and the Exception Rules That Fixed Indexing Loss"}]},{"@type":"WebSite","@id":"https:\/\/unitconversion.io\/blog\/#website","url":"https:\/\/unitconversion.io\/blog\/","name":"Unit Conversion Blog","description":"On conversion and other things :)","publisher":{"@id":"https:\/\/unitconversion.io\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/unitconversion.io\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/unitconversion.io\/blog\/#organization","name":"Unit Conversion Blog","url":"https:\/\/unitconversion.io\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/unitconversion.io\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2021\/01\/uclogo.png","contentUrl":"https:\/\/unitconversion.io\/blog\/wp-content\/uploads\/2021\/01\/uclogo.png","width":500,"height":500,"caption":"Unit Conversion Blog"},"image":{"@id":"https:\/\/unitconversion.io\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/unitconversion.io\/blog\/#\/schema\/person\/4ea06b340c4660f4a04bd6d58c582b69","name":"Olivia Brown","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/unitconversion.io\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/441e8f5d29c2bd1022936f38e27eee93?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/441e8f5d29c2bd1022936f38e27eee93?s=96&d=mm&r=g","caption":"Olivia Brown"},"description":"I'm Olivia Brown, a tech enthusiast and freelance writer. My focus is on web development and digital tools, and I enjoy making complex tech topics easier to understand.","url":"https:\/\/unitconversion.io\/blog\/author\/olivia\/"}]}},"_links":{"self":[{"href":"https:\/\/unitconversion.io\/blog\/wp-json\/wp\/v2\/posts\/7902"}],"collection":[{"href":"https:\/\/unitconversion.io\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/unitconversion.io\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/unitconversion.io\/blog\/wp-json\/wp\/v2\/users\/79"}],"replies":[{"embeddable":true,"href":"https:\/\/unitconversion.io\/blog\/wp-json\/wp\/v2\/comments?post=7902"}],"version-history":[{"count":1,"href":"https:\/\/unitconversion.io\/blog\/wp-json\/wp\/v2\/posts\/7902\/revisions"}],"predecessor-version":[{"id":7908,"href":"https:\/\/unitconversion.io\/blog\/wp-json\/wp\/v2\/posts\/7902\/revisions\/7908"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/unitconversion.io\/blog\/wp-json\/wp\/v2\/media\/3745"}],"wp:attachment":[{"href":"https:\/\/unitconversion.io\/blog\/wp-json\/wp\/v2\/media?parent=7902"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/unitconversion.io\/blog\/wp-json\/wp\/v2\/categories?post=7902"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/unitconversion.io\/blog\/wp-json\/wp\/v2\/tags?post=7902"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}