ProtagorasBot

What it is

ProtagorasBot reads the public RSS, Atom and WordPress feeds of Greek news sites, along with the articles they link to. Protagoras uses this to group coverage of the same story across outlets. The bot runs on a fixed schedule and only reads publicly available pages.

For the media map, ProtagorasBot also opens the pages of the opinion texts of the outlets on the map, under the same rules: it checks robots.txt and makes at most one request per second to each server. Two artificial intelligence models read the text to record which positions its author takes; where the two disagree, a third model reads it. The text is not stored.

How it behaves

• It checks robots.txt before every fetch and honors Disallow rules, using the same User-Agent it fetches with.
• It waits at least one second between article fetches to the same host. If robots.txt sets a Crawl-delay, it waits longer.
• Feeds are polled on a fixed interval, by default every 15 minutes.
• It sends back each publisher's own ETag and Last-Modified validators (If-None-Match / If-Modified-Since), so most feed checks are answered with an empty 304 and no content is re-transferred.

How to recognize it

On every request (robots.txt, feeds and articles) ProtagorasBot identifies with the same User-Agent:

ProtagorasBot/1.0 (+https://protagoras.news/about/bot; contact: info@protagoras.news)

What it stores

By default it stores article metadata only: title, link, publication time, the short summary and the author's name, as the feed itself gives them. It does not republish full article text. Full text is never stored without an explicit agreement with the publisher.

From an opinion text it keeps only a fingerprint of each paragraph, so it can recognise the same text if it is published again, the author's name, and the positions found, each with a short quotation and a link to the article.

Contact

For anything about this bot, email us at info@protagoras.news