Crawlab uses SeaweedFS as its distributed file system.
Below is how it works with files synchronization between the master node and worker nodes.
