WP Content Crawler is dedicated to providing all users with robust, reliable, and automated crawling capabilities. Our goal is to democratize this field, ensuring that these capabilities are accessible beyond developers alone.
WP Content Crawler offers an extensive range of features. You can automatically crawl, update, delete, translate, and spin posts. Additionally, you have the flexibility to specify which fields to translate or spin, granting you precise control. The find-and-replace functionality enables you to modify virtually any content. Furthermore, you can assign information from the target post to a shortcode and place it anywhere within the post.
WP Content Crawler even accommodates WooCommerce products and can save details for third-party plugins that we may not even know. Our features are designed for maximum flexibility, allowing you to tailor them to your requirements. We understand that you may require advanced versions of these features, and we design accordingly. Our commitment extends to maintaining and enhancing the code and features for continuous improvement.
WP Content Crawler Features:
- Recrawl (update) posts
Recrawl posts automatically to keep them updated all the time. You can limit how many times a post can be updated, set update intervals, and ignore old posts. - Delete posts
Do you want to delete old crawled posts? WP Content Crawler can delete them automatically. - Visual Inspector
Just click on an element to find its CSS selector. You can also get alternative CSS selectors that you might be interested in. There is no need to leave your admin panel anymore. - Crawl (scrape, grab, save) posts.
After the settings are configured, WP Content Crawler finds the URLs of the posts and crawls them automatically in the background. - Save every post detail.
Title, excerpt, content, tags, categories, slug, date, custom meta, taxonomies, meta keywords, meta description, featured image, post images, status… Just everything. - Save categories
The target category does not exist on your site? No problem. WP Content Crawler can create the target categories for you. Just define the CSS selectors that find category names. They can even be created as subcategories. - Save slugs (permalink)
You can define the permalink of the posts. You can get the permalink from the target site using shortcodes, enter custom text, and even create templates for the slugs. - Save taxonomies
Save taxonomy values by retrieving them from the target site or entering them manually. Saving details of custom post types is easier than ever. - Control scheduling
WP Content Crawler can set how many times URL collection and post-crawling events should run each time for a site. For instance, you can save three posts every minute or run a URL collection five times every 2 minutes. - Content templates
Use shortcodes to prepare post content, title, excerpt, list item, and gallery item templates. Moreover, you can define templates for the values of each CSS selector using the options box. - Tools
Using the tools, you can save posts manually with their URL, recrawl posts with their ID, or delete already-saved URLs. - Proxy
Use a proxy or proxy to get content from the sites to which your IP does not have access. - Advanced HTML manipulations
Find-replace in response HTML, find and replace in element attributes, exchange element attributes, remove element attributes, manipulate HTML of an element, remove HTML elements, etc.… - Save posts into custom categories.
Does a custom post type have custom categories? No problem. WP Content Crawler can define custom category taxonomies used by the custom post type and select those categories when defining the categories of the post. WP Content Crawler can also create custom categories for you. - Save WooCommerce products
Save price, inventory, shipping, attributes, and advanced options. You can save the product as a simple or an external product. You can also set downloadable file options and define the product as virtual. The options are available for WooCommerce versions greater than or equal to 3.3. - Online Documentation
You can check the online documentation whenever you feel a need. - Ready to translate
You can translate WP Content Crawler into your language using Poedit. - Custom post meta
Save anything as a custom post meta. You can use a CSS selector or type the value. - Automatic translation
Use the artificial intelligence of DeepL Translate API, Google Cloud Translation API, Microsoft Translator Text API, Yandex Translate API, or Amazon Translate API to translate the posts automatically. Note that these are paid services. They generally offer the service for free for a limited amount of time. You can see their pricing pages to learn more.
 pro.png)
