Censorware Design

supren : hejmen
My thoughts on censorware design, excerpted from my posts. Quoted material was written by several other posters on several lists that were carrying this discussion. Think about the early phase of censorware development. If all pages had some sort of rating mechanism, then censorware design would be a trivial problem. But they don't. So what do you do? If you want your design to be rocksolid, then there's only one choice: a page is "bad" unless it has a "good" rating (is the rating trustworthy? that's another issue, a legal issue). The problem? Very few pages are going to pass your test, at least in the beginning, thus making freeware the natural route for properly-designed censorware. But if you want to sell your censorware, directly or indirectly, you're going to have to compromise your design. That's what the censorware vendors have done...
Mardi, 5 Januar 1999, 21:08:05 EST
What little I do know of PICS, and my own experience in computing, and just plain logic, tells me that if the censorware functions properly then a PICS tag is, or should be, an *enabling* mechanism, allowing access to a page. A page without a PICS tag should be blocked. If censorware really does its job, there should be no need for anybody to rate their sites *unless* they wish to enable access for those who use censorware. Mensa has nothing to gain by requiring the ratings.
Mercurdi, 6 Januar 1999, 00:24:36 EST
Ah, now maybe that's the sort of thing we should be thinking about! Of those linked on the Mensa page, which ones do the Right Thing in the default case? Instead of linking to just any censorware vendors, why not say, Hey! this is how censorware should work etc. etc., and these vendors supply censorware that does the Right Thing etc. etc.

This would make the users more knowledgeable, and censorware that doesn't behave properly would have to improve to compete, and we'd help create an environment in which PICS tags on member pages are no longer needed!

Could Mensa be held accountable for incorrect actions taken by faulty censorware? I'm sure some would try, but if a child were to view an inappropriate page because the *censorware* did an illogical thing... Well, it's something to think about.

> We were threatened with a lawsuit...
Threatened? What was the outcome? I don't think they could win. Mensa didn't do anything wrong. The member didn't... (well, I don't know about that - that's not something I'd ever put on a webpage!). The problem is that the *censorware* failed. It assumed that the page was okay because no bad keywords were found, and that's poor design. Though the assumption that a page is okay only if good keywords are found is quite restrictive, it is the *only* way to ensure that the child will be protected (assuming also of course that the keywords can be trusted; but if they were false, that could be used against the creator of the page; but not against Mensa.)
Mercurdi, 6 Januar 1999, 08:42:31 EST
No need to reply concerning irrational censorware.
If it's buggy, it's buggy, and PICS isn't going to help anybody.
Mercurdi, 6 Januar 1999, 21:18:23 EST
> What makes you so sure that is the right behavior? The right
> behavior is whatever behavior the user wants.
What the user wants is for the children to be protected. The only way to insure *that* is to assume that a page is bad if it is not known to be good.

Searching for bad words was a good idea when there were no rating systems and no rated pages, but now it's outdated. Except maybe as an enhancement: if a page fails the badword test, block it even if it does claim to be a good guy via its ratings. (Of course, we all know the problems associated with badword processing.)

If your goal is to protect the children, then you've got to assume that a page is guilty until proven innocent. Would you let somebody not known to be bad take your child away for a trip to the mall? No! That "somebody" must be known to be good!

> You're making a false assumption that the software aims to
> *guarantee* that children will be protected. Software can't
> do that. All it can do, and all it aims to do, is to make
> a "best effort" to filter web content.

I've described a mechanism that would work pretty well. Combine that with severe penalties for falsely enabling inappropriate pages, and you've got a winner.

Jovdi, 7 Januar 1999, 00:25:35 EST
> Many people consider this behavior to be optimal:
>
>   if (picsRating(url) == NULL)
>   {
>     accept = (checkBadWords(url) > badWordsThreshold) &
>              (!isUrlOnBlackList(url))
>   }
>   else
>     accept = (picsRating(url) < ratingThreshold);

But I wouldn't. It's not good enough. For example, how is the BlackList generated? People complained. Why did they complain? Because somebody saw something they shouldn't have. Ergo, the software failed. If I buy censorware, it's because I don't want my kids seeing that stuff, and I don't care what the disclaimer says, they better not see that stuff! Particularly when it's so simple to do it right. If you try this instead, they won't see that stuff...

accept = (picsRating(url) != NULL) && (picsRating(url) < ratingThreshold);

Much cleaner, and faster, and the software is not going to fail. The PICS tag might lie, but that's not the software's fault.

> I have nothing to back me up on this, but I think that *most* people
> would not want to block all unrated sites. They'd only want to
> block unrated sites if they also fail the badWord or BlackList tests.

I disagree. What would you think of Air Traffic Control software that was *designed* to *usually* function properly? And whenever it fails (BOOM!) we just add that misbehavior to the List Of Things To Not Do.

> If that person is not wearing the "I'm good" sign, then what I would
> do (and yes, I am a parent) is subject that person to some additional
> tests.

But would you accept somebody else's test procedures when you *know* that they have failed in the past? And that instead of fixing their test procedures, they merely added those badnames to the BlackList?

Jovdi, 7 Januar 1999, 21:26:47 EST

> The problem with your solution is that many sites do not have the PICS
> tags yet.

That's a problem, and a big one, for the vendor. But it's irrelevant to good program design.

I'll say it again. The customer buys censorware to prevent the children from seeing bad stuff. The only way for screening software to guarantee this is to only show good stuff. Simple, no?

> Again, William, your argument works if most appropriate sites use PICS
> labels, but fails in the current climate.

The current climate is the vendor's problem.

> - in any case, I had originally turned on the blocking of unrated
> sites, and went to let my daughter see her usual sites (disney,
> american girl, etc) - most of them were blocked because they didn't
> have a rating.

Did you complain to the vendor? Did you suggest to her usual sites that they rate their pages? If so, why haven't they? Did you ask for your money back?

Venerdi, 8 Januar 1999, 01:45:15 EST

I said...

I> The only way for screening software to guarantee
I> this is to only show good stuff. Simple, no?

...to which Mr. A replied...

> They do not make that guarantee. Simple, yes?

...to which I reply...

They can't. Because they do not make good software!

Really, it comes down to this. They could have done it right, but the software would have been difficult to sell because, initially, there would be so few sites with access enabled. Instead, they could flip their logic over, throw in some badword processing and a blacklist table, and they'd have a faulty product but one which could function even without many sites enabled. This was a marketing decision, not an engineering decision. And for business purposes, short term and quick return on investment, a wise one. But it's still a poor design, and it's not what the customer wants.

Honestly, if you could choose between screening software that *usually* blocks the bad stuff and screening software that *always* blocks the bad stuff, *and they both always let the good stuff through*, which would you choose? The latter, naturally. That's what you really want.

You object because the software that's available now can't satisfy the *always letting the good stuff through* criterion, and that's because all good pages would have to be rated, and they're not. That's another challenge for the vendor, who then needs to convince people to rate their good pages. As time goes on, more and more pages would be rated, and the software would become more attractive. A better product and a long term strategy.

Lundi, 11 Januar 1999, 23:46:51 EST

> In any case, I don't see the purpose of this whole discussion. Some
> screening software works the way you propose. Some [doesn't]...
> We have to live with what there is. We certainly aren't going to change
> how the software works. So why discuss which way is better?

1) I'm a programmer... it's an uncontrollable urge.

2) Because the Right Thing is for a PICS tag to be an Enabling Mechanism, and if the software does the Right Thing, then one doesn't need to rate a page except to enable access; and if the software does the Wrong Thing, then it is the fault of the software, not Mensa, and we have no liability when faulty software fails to block things that should be blocked. Thus, the PICS requirement is unnecessary.