Plugin is an extension which can extend beyond existing functionalities and features. In Crawlab, the Plugin Framework is in place for users to customize their web crawler management platforms.
# Why Plugin
Why don't we just hack around the source code in Crawlab when customization is needed? The reason is the concern for Maintainability. When you change some code of core modules in Crawlab, you might risk your project's maintainability because there will be upgrades in the future, which would very likely break your current customization.
A well-designed plugin is less likely to be tightly coupled with Crawlab, so that updates in Crawlab will not significantly affect the plugin. Plugins are pluggable and easy to be installed or uninstalled.
# Plugin Framework
Plugin Framework is embedded in Crawlab which manages official and third-party plugins. Crawlab users can develop plugins based on Crawlab Plugin Framework (CPF).
# Official Plugins
There are some public official plugins maintained by Crawlab Team (opens new window). The GitHub repos
of official Crawlab plugins are normally located
in Crawlab Team's repositories (opens new window), each of which has a
|plugin-notification||Sending alerts and notifications such as emails and mobile push notifications||Link (opens new window)|
|plugin-dependency||Installing and managing dependencies and running environment||Link (opens new window)|
|plugin-spider-assistant||Providing advanced web crawler features such as framework support (e.g. Scrapy, etc.)||Link (opens new window)|
# Install Plugin
After a plugin is installed, you should refresh page in your web browser in order for plugin UI components to display.
There are several ways of installing plugins in Crawlab.
# Install Official Plugins
You can install official plugins by only input the plugin name in
Install Plugin dialog.
- Navigate to
Installbutton on plugins you would like to install.
# Install by Git
If you know the git url of a Crawlab plugin, you can install it through git url.
- Navigate to
- Enter the url of the plugin in the field
# Install by Local
This method is recommended only when you are developing Crawlab with source code.
- Navigate to
- Enter local path of the plugin in the field
# Installation Source
Installation Source is only for official plugins.
The default installation source of official plugins is
GitHub. But GitHub is not always the best
Source to access.
For example, if you are in Mainland China, accessing GitHub can sometimes be slow; then you can choose
Source of official plugins, which will largely speed up plugin installation.
# Uninstall Plugin
You can uninstall a plugin by clicking
Delete button on the right in
You can start or stop a plugin by clicking
Stop button on the right in