I don't know if I follow your logic. You want to find a list of all the
sites with a certain keyword in the domain name to block them with a
filter application? A better approach might be to analyze the requested
URL and see if it contains any "bad" keywords. Well, for whatever reason
you want to do this, here is how you can search for sites with a certain
keyword in the URL.
It can be done using GOOGLE. Do a search for inurl:porn for example to
return all sites that have porn somewhere in the url. You might have to
filter it from there to only list sites that have porn before the first
'/' to only include results such as www.examplepornsite.com and not
www.example.com/porn
You can write a perl script to do the google searching for you and then
parse the results. I am not sure if google will agree with this kind of
automatic searching but if you don't hammer their site they shouldn't
complain.
http://www.google.com/search?q=inurl:porn&hl=en&lr=&client=firefox-a&rls
=org.mozilla:en-US:official&start=0&sa=N
that is the URL syntax for searching for sites with "porn" in the URL.
This will return the first page with 10 results. If you want to view
more results you can modify the &start variable for example &start=20
will return results 20 to 30.
The only problem is even if 755,00 matches are found google will only
allow you to view the first 1000, but that could be a good starting
point.
You can then do a search to see sites that link to those sites for
example and even though they don't have porn in the url they are very
likely porn sites, etc.
Regards,
Cosmin Stejerean
-----Original Message-----
From: Vicky Rode [mailto:vicky.rode_at_gmail.com]
Sent: Friday, March 25, 2005 9:36 AM
To: Alexander Chamandy
Cc: vuln-dev_at_securityfocus.com
Subject: Re: Scanner
absolutely NOT but in fact to search for offending sites (porn,
call-home..etc) to be blocked at our filtering appliance.
regards,
/vicky
Alexander Chamandy wrote:
> On Wed, 02 Mar 2005 17:42:24 -0800, Vicky Rode <vicky.rode_at_gmail.com>
wrote:
>
>>Hi there,
>>
>>Just wondering if there is any way I could use a scanner (I have a
home
>>grown script for this) that would go thru the DNS registries from some
>>public source, scan for keywords in the domain name.
>>
>>Will appreciate if someone can point me in the right direction.
>>
>>regards,
>>/vicky
>
>
> You mean to scan whois records for particular domains for keywords in
> the registration information or scan the registry for domain names
> with certain keywords? This wouldn't be used for gathering
> information such as e-mail addresses to spam, would it?
>
>
Received on Mar 28 2005
Intro
