| |
B-News |
|
Miscellaneous
- Linux decoder (can be used with Pan, should
work on Mac OS X too)
Look for the newest version on Stef's
site.
Looking for support?
Despite the fact
that all these support possibilities have been existing for a quite some time
now and all e-mail addresses have been removed from webpages where possible,
some people still seem to believe that lookuing up a developer profile on sourceforge
and harassing him with their questions by mail is the best way to get quick
and accurate support.
All these mails are subsequently shredded, burned, drowned, skinned alive and
finally forwarded to recycle.bin.laden@dev.null and the sender ends up in a
killfile. We don't have time to give anyone individual support by mail, and
we only use live ammo - you're warned.
BommaNews was, in January of '01, the original development title of a newsreading (or rather not) program. It has its roots - development and betatesting - among the users of a Belgian ISP called Telenet (aka Pandora), and Pandora's original logo was a grandma, affectionately nicknamed "bomma" in Flemish, hence the name.
B-News (or bommanews) was developed to lift the weight of the overhead inherent to UUEncode and Base64 encoding: it uses a new encoding method to stuff binary data in text messages. This method eats more CPU resources, but it manages to lower the loss from approximately 40% for UUEncode to 3.5% (the decimal point between those digits is not dirt on your monitor), while still avoiding the use of ANSI control codes in the message body.
For the rest, B-News (the application) is very limited in functionality: it was designed primarily to post files to a usenet server, and download files that were posted in its own format from there again. A stand-alone decoder was also developed to allow downloading with classic newsreaders, and for Mac and Linux users someone else created an encoder and decoder too.
New in version 1.1 are some [very primitive] text posting and reading capabilities.
Find out how B-News and yEnc are related here.
UUEncode and Base64, the encoding methods used by classical newsreaders and mail software, each split three bytes of binary data into four bytes of text, using 6 databits per byte - encoding is done by simply masking bits out of the bytes. Six bits also means that only 64 different characters out of a total of 256 are used in the text representation of binary data, even though more was already theoretically possible in the days when UUEncode and Base64 were developed.
Bommanews pushes it to the limit, and uses all available values that fit in a byte with exception of the ASCII control codes 00 to 1F hex.
Because it uses 224 different codes, this means that it requires a newsserver that treats incoming data in a perfect 8-bit clean way, not only allowing codes above 7F hex, but also not choking on control codes with the parity bit set (80-9F), and it requires that the server gives back exactly the same character sequence when such a message is downloaded later.
Using 224 different codes also means that encoding and decoding isn't any longer a matter of simple logical operations (shift, and, or & the family): in fact the processor is doing quite a bit of arithmetics behind the screens.
Because newsfeeds at telenet's servers usually suck to such an extent that passing out posts would be useless, and because the shared files often aren't interesting for anyone but the persons they are intended to reach, BommaNews posts all its files with a 'local' distribution by default - so the posts never leave the newsserver they are posted on if the server supports this - but not all servers do! This just might be the reason why you haven't seen any yet. [Added: telenet's servers stopped supporting local distribution in June 2001, but instead started filtering out all messages larger than 16K bytes from the newsfeed, which filtering it stopped in turn in september 2002.]
In the mean time, B-News has been tested and has already created a pretty high volume of "messages" without any sign of a problem on three types of newsservers: Microsoft IIS 5.0 and 6.0, and especially on Twister (which happens to be Telenet's newsserver). [Added: this was written in 2001, you can add a bunch to that.]
When it's encoding binary data, B-News first splits these up in 40 byte chunks.
These chunks are regarded as single 320-bit integer numbers (or 80-digit hexadecimal
numbers, or 40-digit numbers in a base-256 numerical system, whichever way you
prefer to regard it).
The encoding process is absolutely simple beyond that: the numbers are converted
to a base-224 representation (pretty much like you could convert hexadecimal
0x1C to decimal 28, only on larger values), and 0x20 is added to each digit
of the result to move it out of the ASCII control code range, so only values
from 20 to FF hex are left.
Why chunks of 40 bytes? Here the answer is simple too: 40 bytes (base-256) is the maximum amount of data that can be represented in base-224 without requiring more than one additional digit: i.e. to encode a block of 40 bytes it takes 41 characters, encoding one of 41 bytes would require 43. By using a 40 byte block size, the encoded data stream is filled to an efficiency of 40/41 = 97.56%, the theoretical maximum efficiency when the entire file would be treated as a single number is 97.59%, or more precisely: log(224)/log(256). The theoretical efficiency of UUEncode, for comparison, is 3/4 = 75%.
At the end of the file, zeroes are appended to fill the last block to 40 bytes.
The original file size is included in the headers, so the decoding process can strip these zeroes back off. The data stream format also allows for other headers, but currently only the filename and size are included.
It is possible to find higher block sizes which would render even less unused space in the encoded stream, but the encoding (and decoding) effort would prove too much for today's processors, and the added efficiency is negligible: 0.03% at most.
The decoding process is the reverse of the encoding process (or what did you expect ;-): subtract 0x20 from every incoming character, divide them in 41-byte blocks, and convert from base-224 to a base-256 representation.
If all of this sounds really simple - whell it is ;-)
The "lot of CPU resources" I mentioned above are the result of the
amount of work it takes to convert a single 40-byte block (converting a 320-bit
integer isn't the same as converting 10 integers of 32 bits each), multiplied
by as many blocks as can be cut out of a file that may be several megabytes
in size - or by the numer of blocks you want to convert per second.
In contrast to most newsreading software, bommanews does the decoding on the fly while the data are coming in from the server: because of the high CPU load, this can cause a noticeable decrease in download speed on slower processors, compared to classic programs on the same high-bandwidth network.
Just to give an idea: on Telenet, Forté Agent reaches speeds of up to 600-650 kBps (when connected to the ISP's server, not some server at the other side of the world). On the same system, a P3/733 with 133 MHz fsb, 256 MB PC133 memory, B-News reaches up to approx. 550 kBps. No need to say that these speeds are reached only in the wee hours of morning when virtually nobody else is on the server ;-) [Added: telenet's average speed has dropped below 100 kBps with peaks below 400, so the difference has become hardly noticeable even on a 133MHz machine]
For slower connection types, the lower amount of data to be transferred weighs through, and up- and downloads will effectively take up to 25% less time.
Especially for server operators: if all your clients use B-News for whatever they're doing in those alt.binaries.* groups, that means 25% less load on the server's disks and 25% less traffic...