Miscellaneous > The Sallyport
"guests"
Sir Edward:
--- Quote from: Sir Edward on 2011-03-11, 15:18:24 ---That's a strange one. Thanks for pointing these out, BTW. Often I can go look and see that they're coming from Asia or something, and block the IP on the server.
EDIT: I'm looking up info regarding that particular message. I'm trying to see if I can figure out what some of these guests are doing that the login info can't show it.
--- End quote ---
Did some digging. Looks like this IP was a crawler called Exabot, which is the crawler for the Exalead search engine. It's a poorly designed one, in that it ignores "robots.txt" and defies other best-practices. Sure enough, the logs show it slogging through the indexes on the forum. But it's also been poking through every available link, like the new-user registration, user profiles, etc. It almost looks like it got stuck, hitting the same links repeatedly. So not really a big deal, it just looks suspicious.
Sir James A:
--- Quote from: Sir Edward on 2011-03-11, 16:04:23 ---
--- Quote from: Sir Edward on 2011-03-11, 15:18:24 ---That's a strange one. Thanks for pointing these out, BTW. Often I can go look and see that they're coming from Asia or something, and block the IP on the server.
EDIT: I'm looking up info regarding that particular message. I'm trying to see if I can figure out what some of these guests are doing that the login info can't show it.
--- End quote ---
Did some digging. Looks like this IP was a crawler called Exabot, which is the crawler for the Exalead search engine. It's a poorly designed one, in that it ignores "robots.txt" and defies other best-practices. Sure enough, the logs show it slogging through the indexes on the forum. But it's also been poking through every available link, like the new-user registration, user profiles, etc. It almost looks like it got stuck, hitting the same links repeatedly. So not really a big deal, it just looks suspicious.
--- End quote ---
If it's ignoring robots, it's growing too complacent and powerful for it's own good, and must be dealt with. :D
Sir Edward:
Heh. :) It may have gotten better. My logs show it looking at robots.txt. Not sure if actually heeds it though. :)
Navigation
[0] Message Index
[*] Previous page
Go to full version