爬虫相关

爬虫相关

爬虫常用框架和工具总结

框架

scrapy、scrapyr-edis

pip install scrapy

pip install scrapy-redis

pip install selenium

页面渲染

selenium

pip install selenium

splash

docker pull splash
pip install scrapy-splash

手机操作

appnium

npm i -g appium
pip install Appium-Python-Client

部署

scrapyd

pip install scrapyd 服务端

pip install scrapyd-client 客户端

gerapy(可视化部署)

pip install gerapy