Neil Schemenauer's Web Log


January 31, 2003

Wolfenstein 5K

A Wolfenstein style game in 5kB. Amazing. So why do the games on my wireless phone suck so bad?

Microsoft software is doubleplus secure

Saw this document referenced from The Reg. I'm happy that MS has decided that security is important. However, they can't just spend a few weeks trying to fix security bugs in millions of lines of code and then claim their software is "secure by design". The document is not even self consistent. First they say things are "secure by design" and "secure by default" but later they say "everyone should stay up to date on patches, use a personal firewall, run antivirus software and keep it up to date".

January 30, 2003

Talking to Americans

Congratulations Canada, on getting a twenty-four hour clock.

That show definitely had its moments. Unfortunately I think Rick Mercer left. Well, I least I get "The Daily Show" here.

January 29, 2003

Ted Turner steps down as AOL/TW VC

The New York-based company earlier on Wednesday posted a loss for all of 2002 of nearly $100 billion, the largest annual loss in U.S. history, after taking a $45.5 billion charge in the fourth quarter to write down the value of assets.

Ouch. I always thought the word synergy was amusing. It sounds cool. Unfortunately it's completely intangible. Goodwill another funny one.

Netscape 4 is my enemy

...and the enemy of anyone who wants to use standard CSS. Back in the day, Netscape designed their own javacript based stylesheet language. When the W3C decided to go with CSS apparently Netscape wrote a hacky JSS->CSS translator. They should have just implemented the parts of CSS that they could do correctly.

Luckily NS4 is going away. Unfortunately there are always those people who are too lazy to upgrade. I found this bit of HTML the other day:

<tr><td style="margin-left: 5px"> 

It crashes Netscape 4.77 (and earlier versions too, not sure which exactly). I have decided not to use it anywhere. It sure is tempting though. Right now I can picture Michael Palin in A Fish Called Wanda yelling "revenge" while driving a steamroller towards Kevin Kline. I hate buggy software.

January 28, 2003

Spam delay experiment updates

My "spam delay" filter has been running for about 9 days now so I decided to summarize the results again. 123 messages were flagged as spam by Spambayes. For 9 of them I whitelisted the sending server. 12 of them were allowed through after 12 hours. Of those 12 only 2 were actually spam. One got to a server via a multi-stage open relay. The other was somehow injected into a server. The rest of the messages were spam. Only 10 of them were retried once or more.

In summary, 100% of legitimate messages were retried for at least 12 hours. Over 90% of spam was not retried. Less than 2% of spam was retried for more than 12 hours. About 4% was retried for more than 6 hours.

January 25, 2003

Microsoft SQL Worm brings Internet to its Knees

The UUnet backbone is hurting. Other ISPs are hurting too:

We're seeing large network disruption here in Los Angeles. Right now, packet loss is running at roughly 95%. (This is not a typo. I do mean *ninety-five percent* packet loss.)

Thank you Microsoft. Software venders need to be liable for product defects. Here's more info on the worm if you are interested. Network administrators, block UDP port 1434 at the router.

January 23, 2003


Bah ha ha. <voice style="wiggum">That's nice work boys</voice>

January 21, 2003

Spam delay results are looking good

My filter has been running for about a day and a half. The results look good. If I leave out legitimate relays (I whitelisted them after the first message), here are the results:

message hash                               first   last  diff tries status
4a895ceca98de1be6f3e9b4732f2f216064c0706   37.5h  37.5h  0.0h     1 denied
4d2672d9bd386c70aadbe7ea884ab8efe3d4892e   37.5h  37.5h  0.0h     1 denied
d9cc82539265c0567a6a72e22303078faf13260f   36.2h  36.2h  0.0h     1 denied
980f9e0800524b8c76077456a1e553d78a620407   32.2h  32.2h  0.0h     1 denied
9979b3990b04dac2b0b0380a844ee984c37229f0   21.7h  21.7h  0.0h     1 denied
2f7c9e891bcea56b2968f0fe91b7b7242239f851   19.5h  19.5h  0.0h     2 denied
5385ba15f4806f835ad8e660fcdb13f857a56647   19.5h   9.7h  9.8h     2 denied
f0fa01a5a4e533f9c9d71322f877a01965d564ed   19.0h  19.0h  0.0h     1 denied
efd1a8de4f1feef125badf5b91f8c3a817f0d9da   12.0h  12.0h  0.0h     1 denied
6bacc0c76eb4c5c8c109887fb19c4f7017f5af7d   12.0h  12.0h  0.0h     1 denied
bcd0da8d26599ce8cc8d0566974d5e7602ff618f    5.1h   5.1h  0.0h     1 denied
fab977d3b64bd5aa4d41f40c20c1f149d982fbb2    4.3h   4.3h  0.0h     1 denied
330281ec24fc962889fb42a2a5b4e9f1db6fee5b    2.5h   2.5h  0.0h     1 denied
c0ebea32075e51a81f6dccac2c6af4ab7ba07f50    2.2h   2.2h  0.0h     1 denied
15a0737682d2212557a350784f6f5e7c0455f9ab    0.7h   0.7h  0.0h     1 denied

The two messages tried twice are being sent from AOL.

Lessons learned? First, most spam is not retried after a temporary error. Second, legitimate relays that relay spam are a problem. I think it's a managable problem though. Whitelists are one solution. A hash cash system would be another. Finally, if the relays used this system the problem would be greatly reduced.

I'm going to continue collecting the data. If anyone wants the code, let me know.

January 20, 2003

Filtering spam without false positives

On the weekend I had an idea for how a spam filter could be installed on a server without causing legitimate email to be blocked. Use a good filter (e.g. Spambayes) and patch it into the mail server so that the spam test is done before the SMTP transaction is done. I'm using qmail but you could use Exim and elspy. If the message looks like spam, reject it with a temporary error (i.e. 4xx) but keep a cookie for it. If the message is still being retried after 12 hours then let it through.

Spammers like to hit and run. They can't wait around for 12 hours because they will be found and shut down. It also costs them or the open relays they abuse to keep retrying. Legitimate email should not be lost; although it could be delayed. Since filters like Spambayes have over 99% accuracy, the amount of mail delayed should be small. Whitelists, perhaps automatically updated, could further reduce the delay of legitimate mail.

I've setup a little test to see how such a system would perform. So far, I have 7 spam messages that were never retried. I have two that where retried a number of times. Those two don't really count because they were forwarded to me from the server (I've now whitelisted that server). I have two more that were retried once. Those two came from an abused AOL account. They have been pending for 3 hours. We will how long AOL kepts trying to send them.

January 15, 2003

So much for the commons

Fucking evil corporations. Robert Heinlein said it best

There has grown up in the minds of certain groups in this country the notion that because a man or corporation has made a profit out of the public for a number of years, the government and the courts are charged with the duty of guaranteeing such profit in the future, even in the face of changing circumstances and contrary public interest.

This strange doctrine is not supported by statue or common law. Neither individuals nor corporations have any right to come into court and ask that the clock of history be stopped, or turned back, for their private benefit.

Copyrights and patents were at one time good things. Now, they have been so twisted by greedy corporations that they are no longer in the public interest. Does any reasonable person think that extending a copyright from 14 years to 150 years increases the incentive for people to create things? Now think about the negative effects of such long protections.

January 14, 2003

"Weapons of mass destruction", huh?

I don't know what changed. Maybe I'm getting old, or maybe it's because I am now seeing mostly US media instead of Canadian, or maybe the world is just changed. "Weapons of mass destruction". Who came up with this term? They must of worked on it for a while. It has obviously been carefully chosen and Bush makes sure to use it liberally.

Perhaps one of it's advantages is that it makes the hypocrisy of the US position a little less apparent (i.e we are allowed to have nukes but you're not). It probably also helps make Saddam look evil (weapons of medium destruction are perhaps okay but weapons of mass destruction are right out).

Also, I really love this one, notice how Bush has carefully twisted things so that it is now Iraq's job to prove they don't have "weapons of mass destruction" instead of the UN's job to prove they do. Proving a negative statement is not easy.

Obviously Saddam is a bad dude and I don't wish for him to obtain nuclear weapons. However, I wish I didn't have to be bombarded by the media with moronic statements (don't even get me started about the Raelian media circus). What do I know though? Bush's approval rating is at 58%. He must be doing something right.


Cults fascinate me. I know it's a tired phrase, but the stupidity of people amazes me. Unfortunately, it's not harmless stupidity. If you are interested in Scientology the book "The Bare-faced Messiah" is an interesting read. Another amusing pass-time is to use "whois" to reveal how much work the cult is willing to do in order to get a high Google ranking.

January 13, 2003

Safari: hurray for standards and competition

There's a lot of buzz about Apple's new web browser Safari. After seeing it in action, I'm impressed. It's fast and the interface is well designed (no surprise since it's an Apple product). Let's hope it helps prod web developers to adhere to standards instead of assuming everyone uses IE. The war is not over yet.

Doc Searls posted an article on the Linux Journal site about it.

Alternative definitions for epoch

This output is human-readable, meaning items like the datestamp are displayed using a textual representation instead of something more obscure (such as the number of nanoseconds since the Tasty Freeze guy drove by).

Creating hybrid CD images that work on Mac and Windows

Use mkisofs -J -r -hfs -map mkisofs_hfs_map, where mkisofs_hfs_map contains something like

# EXTN   XLate   CREATOR   TYPE     Comment
.tif     Raw     '8BIM'    'TIFF'   "Photoshop TIFF image"
.hqx     Ascii   'BnHq'    'TEXT'   "BinHex file"
.doc     Raw     'MSWD'    'WDBN'   "Word file"
.mov     Raw     'TVOD'    'MooV'   "QuickTime Movie"
.txt     Ascii   'ttxt'    'TEXT'   "Text file"
.ppt     Raw     'PPT3'    'SLD8'   "Power Point 98"
.xls     Raw     'XCEL'    'XLS5'   "Excel 5"
.pdf     Raw     'CARO'    'PDF '   "Acrobat File"
.htm     Raw     'MOSS'    'TEXT'   "HTML for Netscape"
.zip     Raw     'ZIP '    'ZIP '   "Zip archive"

Tracking down leaks using the Python gc module

Call the count_types function periodically to see what types are being created.

old_count = {}

def count_types():
    import gc
    new_count = {}
    for o in gc.get_objects():
        k = type(o).__name__
        new_count[k] = new_count.get(k, 0) + 1
    if old_count:
        for k, n in new_count.items():
            old_n = old_count.get(k, 0)
            if n != old_n:
                print k, old_n, '->', n

January 12, 2003

What Control-D really means

While it may appear that Unix C-d means end of file, it actually means "push", and acts exactly like a newline in line-oriented input mode, except that it does not add itself to the end of the input.

Hate flash?

If you are using a Gecko browser (e.g. Mozilla, Netscape 7, Phoenix), you can disable flash content by creating a file {profile}/chrome/userContent.css with the code

/* disable flash content */
embed[type="application/x-shockwave-flash"] {
  display: none !important;

Perl compatible regexes for Common Lisp


Using the hotshot profiler

Hotshot gives more accurate profile results since its overhead is much lower than I have the following code in my $PYTHONSTARTUP file

def hotprofile(func, *args):
    import os, tempfile
    from hotshot import Profile, stats
    filename = tempfile.mktemp(".hotshot")
    p = Profile(filename)
    result = p.runcall(func, *args)
    s = stats.load(filename)
    return result

A PDF version of the Common Lisp Standard

Download the 15.17R files from Create a PDF file using the following commands

dviconcat -o cl-spec.dvi {dvi files in order}
    dvips -Ppdf -G0 cl-spec.dvi -o
    ps2pdf -dMaxSubsetPct=100 -dCompatibilityLevel=1.2 \
        -dSubsetFonts=true -dEmbedAllFonts=true

The resulting PDF file is about 4 MB in size, contains only type 1 (scalable) fonts, and looks good in Acrobat Reader. I think you need both Ghostscript 6 and the type 1 Computer Modern fonts from AMS in order to get good results.

According to Kent Pitman, the draft differs from the official standard only in cover pages, page headers, font size, and whitespace.

D Programming Language

The D Language looks like a fairly well designed systems programming language. I wonder if the compilers are any good.