1. RE: Anyone want to write an "intelligent" mail filter?
- Posted by <jxliv7 at hotmail.com>
Nov 05, 2003
if you guys have been reading /. (slashdot.org) the latest
spammer trick
seems to be DDoS virus/trojans to kill the email
blacklisting (or
whitelisting) sites. i'd hate to be mistaken for one.
bouncing bad email just gives them a means to sell and
address to their
"friends" in the business or it opens you up to a
personalized attack.
when you block an IP or a range of IPs, in most cases you're
blocking a
lot of innocent people along with the bad. and you never
know who might
be tryng to get ahold of you via email -- maybe your 10th
university
reunion, or your granny who got switched ISPs through some
buyout.
the difference between email read with a web browser and
email read with
an email program is going to be a bone of contention for
many, since
there are good points and bad points with each.
i think the best hope for universal email is bayesian
filters, which get
up to the high 90's in efficiency. if the words in the email
fit the
mold that the user determines by email recieved, it's
discarded and
what's left is good email.
all that said, i'm not sure i'd invest the time in writing a
spam
filtering program right now since things are changing too
fast. i'll
just keep my dozen or so accounts where they are and manage
with the 75
or so spams i get a day. that's about 50% spam, by the way.
--
jon
2. RE: Anyone want to write an "intelligent" mail filter?
Rob:
Is there some SPAM filter that not only identifies SPAM, but also avoids
downloading them from the server and deletes them in the server?
I'm asking this because I have a 56K connection, and SPAM consumes a big
part of connection time (I have to pay for it).
A few days ago, I had to resort to change my e-mail address from rforno to
rmforno, in view of increasing SPAM.
Regards.
----- Original Message -----
From: Robert Craig <rds at RapidEuphoria.com>
To: <EUforum at topica.com>
Sent: Wednesday, November 05, 2003 12:11 AM
Subject: Re: Anyone want to write an "intelligent" mail filter?
>
>
> Irv Mullins wrote:
> > Every day I get more annoying SPAM e-mails. Currently it's running about
10
> > spams to every valid e-mail.
> >
> > I'm tired of wading thru them, and I'd rather not download them at all.
> > My e-mail client can filter the messages by sender or subject, but most
> > spams now are written to get around those filters.
> >
> > One thing I notice is that nearly 100% of the spams either contain the
> > word "lagos" or long strings of "dictionary" words to confuse the
filters:
> >
> > "indecisive constitute dakar summitry ajax beaver descendent withal
> > circumlocution asocial voluble inquire convolution replete hitler
> > commendation segregate cognition abstract eject disgustful"
> >
> > But very few or none of the more common shorter words that would likely
> > appear in a valid e-mail: "a, and, or, if, you, we, I, to, for, the,
this,
> > that....."
> >
> > We should be able to come up with a routine which would analyze a given
> > text string and rank it according to its likelyhood of being a
'meaningful'
> > message. Then use that routine in an e-mail client to rank messages and
> > only download from the server those which appear to be 'real'.
> >
> > Ideas?
>
> For the past few months I've been using the e-mail
> client in Netscape 7.1. It has a "Bayesian" spam filter
> that adapts to the streams of spam and normal mail
> that you receive. It works pretty well.
>
> It keeps track of all the words in your incoming e-mail,
> and notes how often each word appears
> in spam vs normal mail. For example, the word "Euphoria"
> might have appeared in 1 of my spam messages and
> 99 of my normal messages, so if it sees "Euphoria" in a
> message, that would indicate a 99% probability that this
> is a normal message. But it doesn't just look at one word.
> I believe it looks at the 20 or so words in each message
> with the most extreme probabilities. It uses a formula from
> Bayesian statistics to combine the probability indicated
> by each word into a single overall probability. e.g.
> if you had a word that indicated "90% likely to be spam"
> and another that said "95% likely to be spam", the result of
> combining those two words might be 97% (or something).
> It will move a message out of your inbox into a spam folder
> if the probability of it being spam is quite high,
> something like 99%. Obviously you want to keep false
> positives (real mail tagged as spam) to an absolute minimum.
>
> In practice, over a long period of time,
> suppose I get 1000 messages of which 900 are spam. It will
> probably move about 800 of the spams and 1 or 2 of the
> non-spams into my spam folder.
>
> With each batch of incoming mail, I check the spam folder
> for non-spams, but usually I can quickly see from the
> subjects and senders that there aren't any non-spams, so I
> click a button to quickly delete all the spams in one
> operation.
>
> Whenever it tags a message incorrectly (usually spam
> that it missed), you can click a button to tell it so.
> This way it gradually learns and gets smarter.
>
> Also, e-mail from anyone in my address book is automatically
> considered non-spam, so the false positives are quite low.
>
> Being able to delete a whole bunch of spams in one
> operation saves time. It's also nice that it keeps
> my inbox largely clear of distracting spam clutter.
>
> Regards,
> Rob Craig
> Rapid Deployment Software
> http://www.RapidEuphoria.com
>
>
>
> TOPICA - Start your own email discussion group. FREE!
>
>
3. RE: Anyone want to write an "intelligent" mail filter?
Ricardo Forno wrote:
>
>
> Rob:
> Is there some SPAM filter that not only identifies SPAM, but also
> avoids downloading them from the server and deletes them in the
> server? I'm asking this because I have a 56K connection, and SPAM
> consumes a big part of connection time (I have to pay for it).
> A few days ago, I had to resort to change my e-mail address from
> rforno to rmforno, in view of increasing SPAM.
> Regards.
I have found that Yahoo's anti-spam features work pretty well, although
in order to access using POP, you'll have to pay about $2 per month.
There is, however, a project at sourceforge that supposedly will access
your Yahoo mail using the web interface, and present a POP interface to
your email client, so you may be able to get around that.
Matt Lewis
4. RE: Anyone want to write an "intelligent" mail filter?
Irv,
Don't know if this helps, but my ISP has a web-based Spam blocker. It's
a free service to all subscribers. You can log on to the Spam blocker,
set the sensitivity, check your Spam "in-box", delete the junk, etc.
I REALLY like it! I can go in, scroll through the Spam and in two
clicks delete everything there. If the Spam blocker traps a legitimate
email, you can tell it to accept all emails from the address in the
future.
I don't know if they bought it or wrote it. You could suggest to your
ISP that they look into getting this software. My ISP is www.yhti.net.
HTH,
Jonas
Irv Mullins wrote:
>
>
> Every day I get more annoying SPAM e-mails. Currently it's running about
> 10
> spams to every valid e-mail.
>
> I'm tired of wading thru them, and I'd rather not download them at all.
> My e-mail client can filter the messages by sender or subject, but most
> spams now are written to get around those filters.
>
> One thing I notice is that nearly 100% of the spams either contain the
> word "lagos" or long strings of "dictionary" words to confuse the
> filters:
>
> "indecisive constitute dakar summitry ajax beaver descendent withal
> circumlocution asocial voluble inquire convolution replete hitler
> commendation segregate cognition abstract eject disgustful"
>
> But very few or none of the more common shorter words that would likely
> appear in a valid e-mail: "a, and, or, if, you, we, I, to, for, the,
> this,
> that....."
>
> We should be able to come up with a routine which would analyze a given
> text string and rank it according to its likelyhood of being a
> 'meaningful'
> message. Then use that routine in an e-mail client to rank messages and
> only download from the server those which appear to be 'real'.
>
> Ideas?
>
> Irv
>
>
> --
> Windows 98 is *NOT* a virus - viruses are small and efficient.
>
>
5. RE: Anyone want to write an "intelligent" mail filter?
Many thanks, Guillermo. I'll try it.
Regards.
----- Original Message -----
From: Guillermo Bonvehi <knixeur at speedy.com.ar>
To: <EUforum at topica.com>
Subject: Re: Anyone want to write an "intelligent" mail filter?
>
>
> Hi Ricardo,
> I had the same problem and i always used http://www.mail2web.com/ and=20
> worked right. It=B4s a free service to check your pop account and delete=
> =20
> spam mails before downloading them.
>
> Hope this helps,
> Guillermo Bonvehi
>
> Ricardo Forno wrote:
> >=20
> >=20
> > Rob:
> > Is there some SPAM filter that not only identifies SPAM, but also avoids
> > downloading them from the server and deletes them in the server?
> > I'm asking this because I have a 56K connection, and SPAM consumes a big
> > part of connection time (I have to pay for it).
> > A few days ago, I had to resort to change my e-mail address from rforno
t=
> o
> > rmforno, in view of increasing SPAM.
> > Regards.
>
>
>
> TOPICA - Start your own email discussion group. FREE!
>
>
6. RE: Anyone want to write an "intelligent" mail filter?
Many thanks. I'll download it.
But, does it avoid downloading spam? Not clear by what you wrote.
Anyway, up to the moment my new account hasn't received any spam, so I will
not be able to test this or other anti-spam software until I start getting
spam again...
Regards.
----- Original Message -----
From: Irv Mullins <irvm at ellijay.com>
To: <EUforum at topica.com>
Subject: Re: Anyone want to write an "intelligent" mail filter?
>
>
> On Wednesday 05 November 2003 11:57 am, Ricardo Forno wrote:
>
> > Rob:
> > Is there some SPAM filter that not only identifies SPAM, but also avoids
> > downloading them from the server and deletes them in the server?
> > I'm asking this because I have a 56K connection, and SPAM consumes a big
> > part of connection time (I have to pay for it).
> > A few days ago, I had to resort to change my e-mail address from rforno
> > to
> > rmforno, in view of increasing SPAM.
>
> I've just downloaded Save My Modem which seems to do exactly what we've
been
> talking about. It identified the first batch of e-mail (6 messages, 4 of
them
> spam) 100% correctly, and offered to delete them from the server and/or
> send "mailbox not found" bounces. It uses SpamAssassin.
>
> It's at http://savemymodem.sourceforge.net
>
> Looks good.
> Irv
>
> --
> Windows 98 is *NOT* a virus - viruses are small and efficient.
>
>
>
> TOPICA - Start your own email discussion group. FREE!
>
>
