The Batch Approach to Networking

by Odd de Presno


Sample text from the Online World Monitor newsletter
ISSN: 0805-6315. June 1994.
(C) by Odd de Presno, Norway.

Links are not maintained! Check the handbook for current links.

Version 1.5 of The Online World book's Chapter 12 has a new section called "Using email gateways to Internet resources". This is a dear topic to me. The text contains tricks that has saved me a lot of time. Actually, it let me do things that I could not have done in other ways.
The most important discovery is The Stanford Netnews Filtering Service. It lets me search through a very large number of Usenet newsgroups each day for the occurrence of selected keywords, and have the results delivered to my mailbox. (Editor's note: This service was later taken over by Reference.COM.)
The following top level newsgroups are currently covered: DU, ca, general, ont, trial, air, can, gnu, out.going, triangle, alt, ieee, rec, tx, atl, comp, in.coming, sci, ucb, aus, control, info, scruz, uk, ba, csd, junk, soc, vmsnet, bionet, ee, misc, su, za, bit, fj, ne, talk, biz, fl, news, test
I have used the filtering service since the 4th of April in connection with these applications:
[1] To track mentions of The Online World handbook on Usenet. Since I assume that the book is often referred to by also giving my name, I use the search term 'presno'. (I fear that the search term 'the online world' is too general, and will result in too many irrelevant hits.) The filtering service is set to give me daily reports.

[2] As the project director of KIDLINK, a large grassroots network promoting a global dialog between youth through 15 years of age, I want to track the search term 'kidlink'. The filtering service is set to give me a report every three days.

[3] Finally, I track the usage of raw ginger used for treatment of my wife's kidney disease. It is widely used in China, and some other Asian countries to help with kidney problems. I simply wants to learn more. The search term is 'ginger'.

Subscribing for regular searches

These services were set up by email to mailto:netnews@db.stanford.edu (Defunct. Today, please check Reference.COM for revised instructions). I put the following set of commands in the _text_ of my mails, and sent one mail per subscription.

    subscribe Presno
    period 1

The "period 1" means that I want daily reports.

    subscribe kidlink
    period 3

    subscribe ginger
    period 3

The search profiles are like queries in WAIS, i.e., plain English text (no boolean AND, OR, NOTs). I have tried to get more details on how the search engine works, but without success. The only thing that I know, besides the results that I get, is what they say: "After you receive useful articles, you can feed them back to the service to improve your profile."
You can also subscribe by World Wide Web using the URL given above.
I have tried this using the Lynx WWW browser, but find the email method quicker, safer, and easier for me.
These simple search terms are used by the Filtering service to search all netnews articles in newsgroups available to the Stanford news host. We are talking BIG numbers here. On a typical day in December, 1993, 43,000 news articles were posted on Usenet representing some 81 megabytes!
All articles that match my profiles (based on content, regardless of which newsgroups they fall into) will be mailed me. Here is a recent example from my 'kidlink' subscription:


 Subscription 2

 Article: comp.sys.mac.comm.31195
 Message-ID: <1994May29.0609.244@afterlife.ncsc.mil>
 From: jpcampb@afterlife.ncsc.mil (Joe Campbell)
 Subject: How to install NinjaTerm/Terminal 
on US Mac? (English)
Score: 80 Lines: 44 First 20 lines: Please excuse my posting in English. I hope that
you can help me.
NinjaTerm or Terminal were recommended to me by the KIDLINK coordinator in Japan. I'm attempting to use one of them for our 5th grade students to communicate over KIDLINK-J with students in Japan. This is my first attempt to set up a computer (Macintosh LC-III) to be Japanese compatible and I'd be most grateful if you could answer a few questions and offer any advise: 1) Is 0.98vx the current version of NinjaTerm and 2.2-J1-0.0.11 the current version of Terminal? If not, can you tell me how to get them? 2) Does NinjaTerm have the capability to store telephone numbers and execute scripts? (Terminal appears to have this capability, but I haven't figured it out yet.) 3) On a US Mac, what is required to operate NinjaTerm or Terminal (I assume fonts are required; I could install Apple's Japanese Language Kit - I think that KanjiTalk would be difficult for us because Article: comp.sys.mac.comm.31225 Message-ID: <2sb1j3$jt6@mudraker.mtholyoke.edu> From: matsuda@mtholyoke.edu (Takeshi Ken Matsuda) Subject: Re: How to install NinjaTerm/Terminal on US Mac? (English) Score: 76 Lines: 70 First 20 lines: Joe Campbell (jpcampb@afterlife.ncsc.mil) wrote: > Please excuse my posting in English. I hope > that you can help me. NinjaTerm or Terminal > were recommended to me by the KIDLINK > coordinator in Japan. I'm attempting to use one > of them for our 5th grade students to communicate > over KIDLINK-J with students in Japan. This is > my first attempt to set up a computer (Macintosh > LC-III) to be Japanese compatible and I'd be most > grateful if you could answer a few questions and > offer any advise: > > 1) Is 0.98vx the current version of NinjaTerm and > 2.2-J1-0.0.11 the current version of Terminal? > If not, can you tell me how to get them? > My version of NinjaTerm is 97a4. Could you tell *me* where you obtained 0.98vx? Also, I once tried Terminal, but that was 2.1x. Would you post where you obtained the above-mentioned version?


Notice the line starting with "Score." The number decreases as the probability of this being a good hit decreases (based on what you have fed back to the service). I have a threshold on the number of hits that I want with each report. The ones with the lowest score numbers are just cut off.
The first hit above has these two lines:
 Lines:  44
 First 20 lines:

This means that the service is set _only_ to give me the first 20 lines of each found message. If I want the full text, then I must send a message back to netnews containing this command:

    get comp.sys.mac.comm.31195

If you wait a couple of days before sending the command, the message may not available any longer. This has happened to me.

Ad hoc searches

You can also do ad hoc searches through the filtering service. For example, to search for articles related to "information filtering," send an email message to the service containing the following command in the TEXT of your mail:

 
     search information filtering

I would have enjoyed knowing how many days' worth of news articles that such commands are searching, but this information seems not to be available.
A while ago, I was writing an article about net sources of information for local area network managers. Using search words like 'banyan', I was able to quickly find many good references and examples.
While this search command is nice to have, I personally find it less useful than the subscription service. After all, there are so many other sources of information, both on CD-ROM (like Computer Select from Ziff-Davis), and online.

Conclusion

I am very satisfied with the The Stanford Netnews Filtering Service. The term 'kidlink' is so unique that I just get what I want. The term 'presno' has given a few strange messages in Bulgarian, but the hit rate must be 99 percent or more.
With the term 'ginger' it is different. Several people have Ginger as a name (the search is not case sensitive), there are internet hosts with ginger in the address, thus giving me irrelevant hits, and ginger is used in so many applications ranging from cooking, a cure for sea sickness, and does also regularly appear in newsgroups about pets (animals).
I have therefore regularly fed useful articles back to the service to improve my search profile on ginger.
You can retrieve instructions on the email interface to the service by sending a message with the word "help" in the message body the service. I keep a copy on my disk for reference. Actually, the gist of the instructions have been incorporated in my communications scripts. When I write a message to the filtering service, a short help text pops up to remind me what to do.
There is a companion service at elib@cs.stanford.edu for filtering computer science technical reports. A search server is also available at URL http://elib.stanford.edu/.

The Online World Monitor newsletter

The newsletter and the book were companions. While the book describes the online world as it is, the newsletter tracked changes. It could more freely focus on selected offerings or phenomena than could be done within the strict framework of the book.
Bulletin 1 had the following contents:

  1. The batch approach to networking.
  2. Spotlight on important developments.
  3. Pointers to Interesting conferences, newsgroups, and resources
  4. Tracking Changes, - or sic transit gloria mundi.

For more about the newsletter, see monitor.html
KIDLINK: http://www.kidlink.org


Feel free to redistribute as long as the text remains intact as it appears here (including this paragraph). Permission to quote/excerpt/reference in other media is hereby granted, so long as cited material is identified as coming from The Online World Monitor newsletter. For any other use, contact the author for permission.

| Index | Expanded index | Register | For Quick Navigation |

Search the handbook:

The Online World resources handbook's text on paper, disk and in any other electronic form is © copyrighted 2000 by Odd de Presno.
Updated at November 15, 2000.
Feedback please.