python3-robot-detection

Detect web crawlers from HTTP UserAgent (Python 3 version)
  https://github.com/rory/robot-detection
  0
  no reviews


This Python module detects if a given HTTP User Agent is a web crawler. It uses the list of registered robots from the robotstxt.org.

This package contains the Python 3 version of the library.