Parsero
Tool and Usage
Parsero is a Python script to analyze robots.txt on web servers. It specifically looks for the Disallow entries and checks which entries might be accessible.
Introduction
Entries that should not be crawled by a web spider, are typically placed in a Disallow entry in the robots.txt file. This file is read by a crawl tool and any of the Disallow entries are skipped for indexing. These entries are interesting, as sometimes they reveal a lot of information about the web server. This tool helps to quickly check which entries are accessible.
Usage and audience
Tool review
The review and analysis of this project resulted in the following remarks for this security tool:
Strengths
- + The source code of this software is available
Author and Maintainers
Parsero is under development by Javier Nieto.
Installation
Support operating systems
Parsero is known to work on FreeBSD, Linux, macOS, and Microsoft Windows.
Parsero alternatives
| Project details | |
|---|---|
| Latest release | 0.81 [2014-09-29] |
| License(s) | GPLv2 |
| Last updated | Sept. 17, 2017 |
Project health
60
This score is calculated by different factors, like project age, last release date, etc.Links
 | Parsero GitHub project |
