close
WordPress.org
Get WordPress
WordPress.org

Plugin Directory

Plugin Tag: crawler

Plugin Tag: crawler

  • BERJAYA

    Email Address Encoder

    (160 total ratings)

    A lightweight plugin that protects email addresses from email-harvesting robots, by encoding them into decimal and hexadecimal entities.

    Till Krüss 100,000+ active installations Tested with 7.0
  • BERJAYA

    Robots.txt Editor

    (8 total ratings)

    Robots.txt for WordPress

    Processby 10,000+ active installations Tested with 5.6.17
  • BERJAYA

    Spider Analyser – WordPress搜索引擎蜘蛛分析插件

    (1 total ratings)

    Spider Analyser是一款用于跟踪WordPress网站各种搜索引擎蜘蛛爬行日志的插件，并进行详细的蜘蛛爬行数据统计、蜘蛛行为分析、蜘蛛爬取分析及伪蜘蛛拦截等。

    wbolt.com 3,000+ active installations Tested with 6.8.5
  • BERJAYA

    No-Bot Registration

    (27 total ratings)

    Prevent bots from creating accounts by blacklisting domains and usernames and present people with a human friendly security question.

    Arnan 2,000+ active installations Tested with 6.9.4
  • BERJAYA

    Robots.txt rewrite

    (2 total ratings)

    Provide the easy managment of your robots.txt from admin side. It propose you the advanced then standard robots.txt content too.

    Eugen Bobrowski 1,000+ active installations Tested with 4.7.33
  • BERJAYA

    Marquee image crawler

    (1 total ratings)

    Marquee image crawler is a continuous scrolling image plugin. This plugin crawls the images left or right.

    gopiplus 700+ active installations Tested with 6.1.10
  • BERJAYA

    Mass Ping Tool for SEO – WordPress ping list to get indexed faster on Google, Yandex, …

    (4 total ratings)

    Mass ping tool for SEO plugin is one of the fastest ways to get indexed by Google. After pinging, Google bots, crawlers will visit your website and in …

    Pagup 500+ active installations Tested with 6.9.4
  • BERJAYA

    Bisteinoff SEO Robots.txt

    (4 total ratings)

    An easy-to-use plugin that generates and configures a proper robots.txt file, essential for effective search engine optimization (SEO).

    Denis Bisteinov 500+ active installations Tested with 6.9.4
  • BERJAYA

    Block Archive.org via WordPress robots.txt

    (0 total ratings)

    Blocks the archive.org bots through the WordPress virtual robots.txt file.

    apasionados 500+ active installations Tested with 6.9.4
  • BERJAYA

    bodi0`s Bots visits counter

    (5 total ratings)

    Counts the visits from web spiders, crawlers and bots in your blog, with ability to get the blog rankings.

    Budiony Damyanov 100+ active installations Tested with 6.1.10
  • BERJAYA

    Block Chat GPT via robots.txt

    (1 total ratings)

    Blocks the ChatGPT-User bot that is used by plugins in ChatGPT to crawl websites through the WordPress virtual robots.txt file

    apasionados 100+ active installations Tested with 6.3.8
  • BERJAYA

    Block eCommerce Assets via robots.txt

    (0 total ratings)

    Blocks some WooCommerce assets and Search pages that should not be indexed through robots.txt

    apasionados 100+ active installations Tested with 6.3.8
  • BERJAYA

    AI Deny

    (1 total ratings)

    Easily block over 30 of the most common AI user-agents known for crawling websites, keeping unauthorized bots at bay.

    Code For Broke 70+ active installations Tested with 6.9.4
  • BERJAYA

    Warmer – Cache Warmer and Cache Preload Crawler

    (4 total ratings)

    Warm your WordPress cache with scheduled crawling, cache preloading, URL source control, logs, diagnostics, and WP-CLI.

    Atakan Oz 60+ active installations Tested with 7.0
  • BERJAYA

    Findkit – Site Search

    (0 total ratings)

    Powerful cloud based search engine for your website. Search as you type, real-time index updates, and embeddable UI. Free for small websites.

    Findkit Team 60+ active installations Tested with 6.9.0
  • BERJAYA

    Block Feed and Comments via robots.txt

    (0 total ratings)

    Blocks some Feed and Comment resources in order to avoid spam bots

    apasionados 30+ active installations Tested with 6.3.8
  • BERJAYA

    Block Common Crawl via robots.txt

    (0 total ratings)

    Blocks the Common Crawl bot (CCBot) that gathers data that is used to train AI algorithms like ChatGPT through the WordPress virtual robots.txt file

    apasionados 30+ active installations Tested with 6.3.8
  • BERJAYA

    Azayem Bots Tracker – Bot Visits Logger

    (0 total ratings)

    A custom plugin to log search engine bot visits and analyze their crawl budget directly inside WordPress (no external services required).

    botstracker 10+ active installations Tested with 6.9.4
  • BERJAYA

    Robots.txt Extender

    (0 total ratings)

    Dynamic robots.txt for Multisite! Change parameters, or don't, for each site of your network without losing the defaults from WordPress Includes.

    bettoadami 10+ active installations Tested with 5.0.25
  • BERJAYA

    JaspreetChahal’s wordpress bot detector lite

    (1 total ratings)

    This plugin detects a few bots (e.g. Google, Google ads, Alta vista etc) and sends an email based on interval set by you. you can choose what format e …

    Jaspreet Chahal 10+ active installations Tested with 4.2.39