From bhern at netscape.com Sun Nov 2 14:07:25 1997 From: bhern at netscape.com (Brian Hernacki) Date: Sun Dec 7 20:19:21 2003 Subject: nntp-extensions Renaming the SEARCH extensions References: <199710311851.KAA21322@imail2.microsoft.com> Message-ID: <345CF99D.4384546F@netscape.com> Nat Ballou wrote: > It turns out that Microsoft will ship in the next week an > implementation of the search draft that will use the > extension name SEARCH. This release has > been out in beta for several months using the SEARCH > extension name, since this has been the extension > name used in all versions of the search draft. > > As I'm sure there will be other changes to the search > draft both as a result of RFC977bis work as well as > discussions of the search draft on the full mailing list, I view > this as a minor issue. Specifically, both Netscape and > Microsoft will likely have to release another update to cover > changes as a result of these discussions. Sounds like the same lines I was thinking along. We'll update our server and clients once SEARCH is official. Unless some really objects, let's change the name of the extension and reccomend that no one implement with that name until the draft is finalized. --brian From natba at MICROSOFT.com Mon Nov 3 09:02:59 1997 From: natba at MICROSOFT.com (Nat Ballou) Date: Sun Dec 7 20:19:21 2003 Subject: nntp-extensions Renaming the SEARCH extensions Message-ID: <0158049011703b7PURINA@ims.microsoft.com> Brian, The reason we put search in the latest release of our product was so that client developers could have a server to hit against. Specifically, the NNTP service bundled with beta 3 of the NT Option Pack shipped a couple of months ago included search support. While it is already out of date (i.e., it accepts the optional charset argument- which has been eliminated in the latest search draft), I don't view this as a big deal until there is something more than a draft on search. The real purpose is to get clients thinking about search so that they contribute to the search discussion. Clients aren't likely to include search support until we get a little further with the search. Also, getting a syntax error against the Netscape server when you use the 'SEARCH' extension name does not seem like a big deal to me - probably as good as any other error you can give until Netscape provides an implementation of the search draft. So, I'd prefer to leave the draft with 'SEARCH' as the extension name. Given the current rate of progress on RFC977bis, I think Netscape will have plenty of time to update their implementation. Nat P.S. - if you also want developers to hit against a Netscape implementation but have problems with release dates, how about putting up a early version of the Netscape server on the Netscape Internet site that does support the search draft? Microsoft will probably do this in the next month or so ... -----Original Message----- From: Brian Hernacki To: Nat Ballou Cc: nntp-extensions@academ.com Date: Sunday, November 02, 1997 2:27 PM Subject: Re: nntp-extensions Renaming the SEARCH extensions >Nat Ballou wrote: >> It turns out that Microsoft will ship in the next week an >> implementation of the search draft that will use the >> extension name SEARCH. This release has >> been out in beta for several months using the SEARCH >> extension name, since this has been the extension >> name used in all versions of the search draft. >> >> As I'm sure there will be other changes to the search >> draft both as a result of RFC977bis work as well as >> discussions of the search draft on the full mailing list, I view >> this as a minor issue. Specifically, both Netscape and >> Microsoft will likely have to release another update to cover >> changes as a result of these discussions. > >Sounds like the same lines I was thinking along. We'll update our server >and clients once SEARCH is official. > >Unless some really objects, let's change the name of the extension and >reccomend that no one implement with that name until the draft is >finalized. > >--brian From bhern at netscape.com Mon Nov 3 10:32:47 1997 From: bhern at netscape.com (Brian Hernacki) Date: Sun Dec 7 20:19:21 2003 Subject: nntp-extensions Renaming the SEARCH extensions References: <0158049011703b7PURINA@ims.microsoft.com> Message-ID: <345E18CF.25D4D08F@netscape.com> Nat Ballou wrote: > Clients aren't likely to include search support until > we get a little further with the search. Also, getting a > syntax error against the Netscape server when you > use the 'SEARCH' extension name does not seem > like a big deal to me - probably as good as any other > error you can give until Netscape provides an > implementation of the search draft. > > So, I'd prefer to leave the draft with 'SEARCH' as > the extension name. Given the current rate of progress > on RFC977bis, I think Netscape will have plenty > of time to update their implementation. It's not a matter of releasing a up-to-spec client or server. It's a matter of deployed clients and servers. Just because new software is available doesn't mean people will upgrade to it... nor should we force them to. If we leave it as SEARCH we're going to see a lot of breakage. It's seems a pretty simple thing to do to solve interoperability problems and I can't see any downside to it. > P.S. - if you also want developers to hit against a > Netscape implementation but have problems with > release dates, how about putting up a early version > of the Netscape server on the Netscape Internet > site that does support the search draft? Microsoft > will probably do this in the next month or so ... We already do. --brian Opinions are mine, not Netscape's From natba at MICROSOFT.com Mon Nov 3 11:06:51 1997 From: natba at MICROSOFT.com (Nat Ballou) Date: Sun Dec 7 20:19:21 2003 Subject: nntp-extensions Renaming the SEARCH extensions Message-ID: <019db17051903b7PURINA@ims.microsoft.com> > From: Brian Hernacki > Subject: Re: nntp-extensions Renaming the SEARCH extensions > > [...] > >If we leave it as SEARCH we're going to see a lot of breakage. It's >seems a pretty simple thing to do to solve interoperability problems and >I can't see any downside to it. I guess this is the part I don't understand. What clients will break? Are they only Netscape clients? Nat From bhern at netscape.com Mon Nov 3 11:12:18 1997 From: bhern at netscape.com (Brian Hernacki) Date: Sun Dec 7 20:19:21 2003 Subject: nntp-extensions Renaming the SEARCH extensions References: <019db17051903b7PURINA@ims.microsoft.com> Message-ID: <345E2212.F7CC4B27@netscape.com> Nat Ballou wrote: > > > From: Brian Hernacki > > Subject: Re: nntp-extensions Renaming the SEARCH extensions > > > > [...] > > > >If we leave it as SEARCH we're going to see a lot of breakage. It's > >seems a pretty simple thing to do to solve interoperability problems and > >I can't see any downside to it. > > I guess this is the part I don't understand. What clients will > break? Are they only Netscape clients? Hopefully it's only Netscape. But I had a couple of developers contact me several months ago to get the specs for how we implementing search at the time so it could be more. Either way though, you have several hundred servers (at least) out there and many thousands of clients who speak a different SEARCH than what the spec is doing. I know protocol-purity says "let 'em break" but I can't see doing that when the solution is so simple and easy. --brian Opinions are mine, not Netscape's From natba at MICROSOFT.com Thu Nov 13 09:58:00 1997 From: natba at MICROSOFT.com (Nat Ballou) Date: Sun Dec 7 20:19:21 2003 Subject: nntp-extensions Re: ietf-nntp anti-spam nntp extensions Message-ID: <024f22357170db7PURINA@ims.microsoft.com> > From: Clive D.W. Feather > To: Stan Barber > Cc: jeff.garzik@spinne.com ; ietf-nntp@academ.com > Date: Thursday, November 13, 1997 9:46 AM > Subject: Re: ietf-nntp anti-spam nntp extensions > > [...] > >However, I didn't think we had a standard way to specify or register >extensions yet. I've written the search draft so that it identifies the extension name, with no details on how the extension mechanism works - leaving that to RFC977bis. I think this is sufficient for a draft. Nat From jeff.garzik at spinne.com Sat Nov 29 02:32:53 1997 From: jeff.garzik at spinne.com (Jeff Garzik) Date: Sun Dec 7 20:19:22 2003 Subject: nntp-extensions Anti-spam NNTP extension draft Message-ID: <347FC525.4B5029FC@spinne.com> This is available on the Web at http://www.spinne.com/usenet/prefilter/ -- Jeff Garzik Typhoon, Cyclone, Diablo, and INN Spinne USENET News Service Fast and efficient news feeds http://www.spinne.com/usenet/ News tuning and consulting -------------- next part -------------- NNTP Pre-filtering Provisions for real-time distribution of unwanted article information In order to keep up with the rising tide of spam, real-time transmission of spam message ids to downstream sites is needed. To that end, I am designing anti-spam extensions to NNTP. Here's what I've come up with: 1.1 The MODE FILTER command MODE FILTER MODE FILTER is used by a peer to indicate to the server that it would like to advise the server, using the FILTER command, of message ids the peer considered unwanted. This command should always be used before FILTER. 1.2 Responses 203 Filtering is OK 500 Command not understood 2.1 The FILTER command FILTER filter-tag FILTER is used by a peer to indicate that the peer has processed the article identified by , and advises the server it should consider the article cancelable. filter-tag is a short string identifying the process used on the peer to filter the article. The set of legal characters in the tag is limited to non-whitespace printable ASCII characters. In order to reduce conflicts between differing filter standards, implementors are encouraged to prepend the organization name to the beginning of the filter-tag. This command is merely an advisory notice sent from the peer to the server. The server is free to take any action upon receipt of a FILTER command, including no action at all. 2.2 Responses No response is sent back to the peer when a FILTER command is received. ---------------------------------------------------------------------------- 3.1 Comments As implemented in INN, the following logic would be applied to each passed to the server: if ( -in-history-database ) attempt article cancel else add to history database Smarter implementations could also optimize by, and for, the presence of spam cancel messages. If a spam cancel is present in the form of in history, further processing could be avoided. And if is not in the history database, then can be added to the database as well, to refuse the spam cancel that's almost certainly to follow. 4.1 INN sample implementation PATCH to INN 1.7 (2k text) http://www.spinne.com/usenet/prefilter/innd-prefilter.patch 5.1 Diablo sample implementation PATCH to Diablo 1.13 (1k text) http://www.spinne.com/usenet/prefilter/diablo-prefilter.patch ---------------------------------------------------------------------------- Jeff Garzik Spinne USENET News Service November 14, 1997 From ccaputo at alt.net Sat Nov 29 11:34:49 1997 From: ccaputo at alt.net (Chris Caputo) Date: Sun Dec 7 20:19:22 2003 Subject: nntp-extensions Anti-spam NNTP extension draft In-Reply-To: <347FC525.4B5029FC@spinne.com> Message-ID: Hi. In section 2.2, why is no response sent back? I am not sure if leaving the lock step approach is a good idea. If latency is an issue, I wonder if the following should be done instead or available as an alternative.: >> FILTER filter-tag [similar to post] << nnn Ok (or "nnn fail") [ack/nack filter request] >> [ids to filter...] >> >> [...] >> >> >> >> . [indicate completion, like post] << nnn Ok (or "nnn fail") [ack/nack completion] Assuming there are multiple message-ids per filter-tag, this would decrease the number of bytes exchanged. Chris On Sat, 29 Nov 1997, Jeff Garzik wrote: > This is available on the Web at > > http://www.spinne.com/usenet/prefilter/ > > -- > Jeff Garzik Typhoon, Cyclone, Diablo, and INN > Spinne USENET News Service Fast and efficient news feeds > http://www.spinne.com/usenet/ News tuning and consulting > From jeff.garzik at spinne.com Sat Nov 29 14:49:00 1997 From: jeff.garzik at spinne.com (Jeff Garzik) Date: Sun Dec 7 20:19:22 2003 Subject: nntp-extensions Anti-spam NNTP extension draft References: Message-ID: <348071AC.4A995C59@spinne.com> Chris Caputo wrote: > Hi. In section 2.2, why is no response sent back? I am not sure if > leaving the lock step approach is a good idea. If latency is an issue, I > wonder if the following should be done instead or available as an > alternative.: [...] That's a good question. No response is sent back because of how I plan for this to be implemented on the server side. I don't intend for there to be a large number of sites sending FILTER commands (although such is possible). I envision a few, large anti-spam servers running their filters full tilt, and then pumping out the filtering message ids to hundreds (or thousands) of sites at the same time. Since we're using TCP here, we know either (a) a message id was sent, or (b) the connection was lost. Given that, I didn't see any need to incur the processing overhead on the FILTER server to keep a queue of backlogged message ids (in case a negative response is returned). Once you start backlogging ids, you add latency (the very thing this is trying to reduce). The FILTER server simply sends out message ids to all connected clients. If a client (downstream site) can't be contacted, those FILTER commands to that particular site are simply lost. I would agree that a response is necessary if the FILTER feed is a site's sole means for despamming. The intention of FILTER is not a full and complete anti-spam solution, but rather a fast, quick solution that will beat spam to leaf nodes. -- Jeff Garzik Typhoon, Cyclone, Diablo, and INN Spinne USENET News Service Fast and efficient news feeds http://www.spinne.com/usenet/ News tuning and consulting From jeff.garzik at spinne.com Sat Nov 29 14:50:23 1997 From: jeff.garzik at spinne.com (Jeff Garzik) Date: Sun Dec 7 20:19:22 2003 Subject: nntp-extensions Anti-spam NNTP extension draft References: Message-ID: <348071FF.3A6B2099@spinne.com> Chris Caputo wrote: > Hi. In section 2.2, why is no response sent back? I am not sure if > leaving the lock step approach is a good idea. If latency is an issue, I > wonder if the following should be done instead or available as an > alternative.: Quick thought to add to my last message -- NNTP streaming leaves behind the lock-step approach as well. That's mainly why I added 'MODE FILTER' -- so that there was a clear point in time where the lock-step approach is left behind. -- Jeff Garzik Typhoon, Cyclone, Diablo, and INN Spinne USENET News Service Fast and efficient news feeds http://www.spinne.com/usenet/ News tuning and consulting From ccaputo at alt.net Sat Nov 29 12:15:24 1997 From: ccaputo at alt.net (Chris Caputo) Date: Sun Dec 7 20:19:22 2003 Subject: nntp-extensions Anti-spam NNTP extension draft In-Reply-To: <348071FF.3A6B2099@spinne.com> Message-ID: On Sat, 29 Nov 1997, Jeff Garzik wrote: > Quick thought to add to my last message -- NNTP streaming leaves behind > the lock-step approach as well. That's mainly why I added 'MODE FILTER' > -- so that there was a clear point in time where the lock-step approach > is left behind. Ahh! I think that's good and makes non-lock-step okay. Is it anticipated that downstream sites will propagate the filter information they receive? Chris From jeff.garzik at spinne.com Sat Nov 29 16:31:56 1997 From: jeff.garzik at spinne.com (Jeff Garzik) Date: Sun Dec 7 20:19:22 2003 Subject: nntp-extensions Anti-spam NNTP extension draft References: Message-ID: <348089CC.164859F3@spinne.com> Chris Caputo wrote: > Is it anticipated that downstream sites will propagate the filter > information they receive? Not really. The FILTER protocol was essentially designed as an out-of-band mechanism. Propagation of FILTER commands would lead to a point of diminishing returns. If you start propagating the FILTER commands (which is technically feasible), then you'll eventually reach a point where FILTER commands and cancel messages will be arriving about the same time. -- Jeff Garzik Typhoon, Cyclone, Diablo, and INN Spinne USENET News Service Fast and efficient news feeds http://www.spinne.com/usenet/ News tuning and consulting