Re: Correctly dealing with bots and scrapers.

["Constantine A. Murenin via NANOG" <nanog () lists nanog org>]

Hi Andrew,

Yes, you could use something like the following with nginx.conf:

  location ^~ /wp- { return 444; }

The `^~` modifier will ensure that the regex locations will not be checked.

The 444 return is a special nginx code that does a connection shutdown
without sending a response, this may tie up the resources of the bot
doing the scans.

References:

* http://nginx.org/r/location
* http://nginx.org/r/return

Best regards,
Constantine.

On Thu, 17 Jul 2025 at 12:07, Andrew Latham <lathama () gmail com> wrote:
> Constantine
>
> Good call there, I need to investigate the 404 responses to see if
> there are any improvements to be made.
>
> On Wed, Jul 16, 2025 at 11:22 PM Constantine A. Murenin
> <mureninc () gmail com> wrote:
> > On Wed, 16 Jul 2025 at 14:33, Andrew Latham via NANOG
> > <nanog () lists nanog org> wrote:
> > > related topic. Security Scans. Any requests for wordpress could be an
> > > easy way to flag and block with fail2ban when wordpress is not in use.
> >
> > For WordPress and PHP, I think it's simply easier to catch the
> > scenarios with a nginx config, and cheaply return errors from the
> > front end webserver, without wasting any of the real backend
> > resources.
> >
> > C.
_______________________________________________
NANOG mailing list 
https://lists.nanog.org/archives/list/nanog () lists nanog org/message/JNDHS5YJIYASWB7NWE3YSIDEG7TY6RA2/