Suspicious bot ad crawlers from third parties visiting my site

Gary l

May 25, 2025 16:06

My wordfence firewall has blocked the ip ranges of a few bot crawlers that have been visiting suspicious pages on my site like my ads.txt page and the xmlrpc.php page of my site.

Some of the bots are coming from the following companies:

Trade Desk ads.txt & sellers.json crawler

https://www.comscore.com/Web-Crawler

SirdataBot

weborama-fetcher

I'm wondering if these are webcrawlers from journey and mediavine and if I should unblock the ip range they are coming to my site from?

Comments

1 comment

Support

Journey Support Team

May 28, 2025 16:31
Gary l - Typically, any requests to your ads.txt domain will be from advertisers, as they use this domain to verify the identity of your site. Implementing a firewall on this domain is not recommended, as this would cause a potential loss of revenue as advertisers would not be able to access your ads.txt file.
0

Journey is Self Supported

Our support staff is monitoring this forum everyday for bugs related to ad delivery, the display and functionality of the Journey dashboard, and any bugs related to the onboarding process. We are typically unable to assist with non-bug related issues like RPM performance expectations, issues setting up the ads.txt file, and questions already answered by our existing help docs. Read here for a more detailed breakdown of how Journey is self-supported and our recommendations for how to find answers to common issues.

Please sign in to leave a comment.