Home page logo

Politech mailing list archives

FC: Filtering advocate responds to Consumer Reports article
From: Declan McCullagh <declan () well com>
Date: Wed, 14 Feb 2001 23:52:27 -0500

[David Burt is a renegade librarian who used to operate filteringfacts.org and is an ardent defender of the software. Now he works for a censorware vendor. I'm copying Consumer Reports in case they choose to reply to the earlier message (http://www.politechbot.com/p-01733.html). ---Declan]


From: "David Burt" <dburt () n2h2 com>
To: "Declan McCullagh" <declan () well com>
Subject: re:FC: Consumer Reports gives thumbs-down to smut-blocking software
Date: Wed, 14 Feb 2001 20:49:16 -0800
MIME-Version: 1.0
Content-Type: multipart/alternative;
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal

Unfortunately, the Consumer Reports test suffers from the same deficiencies as most other filter tests:

1) An absurdly small sample.
A filter is trying to block no more than 10% of the web at most. With millions of websites, obviously you need a large sample, at a bare minimum 10,000 unique URLs, to get a decent idea of its effectiveness. The Consumer Reports test used only 86 sites. That's simply not enough, the possibility of error is large. When a tester is trying to measure over inclusiveness in multiple categories, "sexually explicit content or violently graphic images, or that promote drugs, tobacco, crime, or bigotry", with 86 sites, absurd doesn't even begin to accurately describe this "test."

2) Purposefully selected sample.
The problem of a small sample is compounded by the fact that the sample is not random. The author of the report obviously had a bias against filters, and using a purposefully selected sample under these conditions is a serious invitation to abuse.

4) Testing the wrong thing
The claims about wrongly blocked sites are made against "AOL Young Teen", which is a white list. That means it is a list of pre-approved sites, rather than a list of excluded sites. The "AOL Young Teen" setting blocks probably 99% of the Internet. It's not something that a school would use as a filter.

5) Drawing the wrong conclusions:
The article discusses filtering requirements for certain schools and libraries. Yet it doesn't test the filters used in schools and libraries. The most popular products in ed space, CyberPatrol Server version, N2H2, I-Gear, WebSense, Smart Filter, and X-Stop were not included in the testing.

This badly conducted research that didn't even test institutional grade filters cannot be said to have any relevance to the appropriateness of filters in schools and libraries.

David Burt, Market Research Manager
dburt () n2h2 com  http://www.n2h2.com/
Intelligent Technologies For A Safe and Productive Internet
Phone 206 892-1130  Fax: 509 271-4226

POLITECH -- Declan McCullagh's politics and technology mailing list
You may redistribute this message freely if it remains intact.
To subscribe, visit http://www.politechbot.com/info/subscribe.html
This message is archived at http://www.politechbot.com/

  By Date           By Thread  

Current thread:
  • FC: Filtering advocate responds to Consumer Reports article Declan McCullagh (Feb 15)
[ Nmap | Sec Tools | Mailing Lists | Site News | About/Contact | Advertising | Privacy ]