storage.conf - configuration file for storage manager
DESCRIPTION
The storage manager is a
unified interface between INN and a variety of different storage method,
allowing the news administrator to choose between different storage method
with different tradeoffs (or even use several at the same time for
different newsgroups or articles of different sizes). The rest of INN
need not care what type of storage method was used for a given article;
the storage manager will figure this out automatically when that article
is retrieved via the storage API.
<pathetc in inn.conf>/storage.conf
file contains the rules to be used in assigning
articles to different storage methods.
The file consists of a series of storage method entries.
Blank lines and lines beginning with a number sign (``#'') are ignored.
The maximum number of character in each line is 255.
The order of entries in this file is important.
Each entry specifies a storage method and a set of rules. Articles that
match all of the rules of a storage method entry will be stored using that
storage method. If an article matches multiple storage method entries,
the first one will be used. Each entry is formatted as follows:
If spaces or tabs are included in a value, that value must be quoted
with ``"''.
If either ``#'' or ``"'' are meant to be included verbatim in a value,
they should be escaped with ``\''.
<methodname> is the name of a storage method to use for articles that
match the rules of this entry. The currently available storage methods
are
``timecaf'', ``timehash'', ``cnfs'', ``tradspool'' and ``trash''.
See the STORAGE METHODS section below for more details.
The meanings of the keys in each entry are as follows:
class
An identifier for this storage method entry. <storage_class> should be a
number and should be unique across all of the entries in this file. It's
used mainly by
expire.ctl(5)
for specifying expiration times by storage class.
newsgroups
What newsgroups are stored using this storage method. <wildmat> is a
wildmat(3)
pattern that is matched against the newsgroups an article is posted to.
If ``storeonxref'' in inn.conf is ``true'', this pattern will be matched
against the newsgroup names in the ``Xref'' header; otherwise, it will be
matched against newsgroup names in the ``Newsgroups'' header. Poison
wildmat expressions (expressions starting with ``@'') are allowed and can
be used to exclude certain group patterns. ``!'' cannot be used, however.
The <wildmat> pattern is matched in order. There is no default newsgroups
pattern; if an entry should match all newsgroups, use an explicit
``newsgroups: *''.
size
A range of article sizes (in bytes) that should be stored using this
storage method.
If <maxsize> is ``0'' or not given, the upper size of articles is limited
only by ``maxartsize'' in
inn.conf(5).
The ``size'' field is optional and may be omitted entirely if you want
articles of any size (that otherwise fulfill the requirements of this
storage method entry) to be stored in this storage method.
expires
A range of article expiration times that should be stored using this
storage method. Be careful; this is less useful than it may appear at
first. This is based
only
on the ``Expires'' header of the article, not on any local expiration
policies or anything in
expire.ctl(5)!
If <mintime> is non-zero, then this entry will not match any article
without an ``Expires'' header. This key is therefore only really useful
for assigning articles with requested longer expire times to a separate
storage method. <mintime> and <maxtime> are boundaries on the amount of
time into the future the ``Expires'' header of the article requests that
it remain around, and are formatted 0d0h0m0s (days, hours, minutes, and
seconds into the future). If <maxtime> is ``0s'' or is not specified,
there is no upper bound on expire times falling into this entry (note that
this key has no effect on when the article will actually be expired, only
on whether or not the article will be stored using this storage method).
This field is also optional and may be omitted entirely if all articles
with or without an ``Expires'' header (that otherwise fulfill the
requirements of this storage method entry) should be stored according to
it.
options
This key is for passing special options to storage methods that require
them (currently only ``cnfs''). See the STORAGE METHODS section below for
a description of its use.
If an article matches all of the constraints of an entry, it is stored via
that storage method and is associated with that <storage_class>. This
file is scanned in order and the first matching entry is used to store the
article.
If an article doesn't match any entry, either by being posted to a
newsgroup that doesn't match any of the <wildmat> patterns or by being
outside the size and expires ranges of all entries whose newsgroups
pattern it does match, the article is not stored and is rejected by
innd(8).
When this happens, the error message
cant store article: no matching entry in storage.conf
is logged to syslog. If you want to silently drop articles matching
certain newsgroup patterns or size or expires ranges, assign them to the
``trash'' storage method rather than having them not match any storage
method entry.
STORAGE METHODS
Currently, there are four storage methods available. Each method has its
characteristics. You can choose any of them to be suitable for your
environment. Note that each method has an attribute ``EXPENSIVESTAT'' which
means whether checking existense of article is expensive or not. This is used
to run
expireover(8).
cnfs
The ``cnfs'' storage method stores articles in large cyclic buffers (CNFS
stands for Cyclic News File System). It's by far the fastest of all
storage methods (except for ``trash''), since it eliminates the overhead
of dealing with a file system and creating new files. Articles are stored
in CNFS buffers in arrival order, and when the buffer fills, it wraps
around to the beginning and stores new articles over top of the oldest
articles in the buffer. The expire time of articles stored in CNFS
buffers is therefore entirely determined by how long it takes the buffer
to wrap around, which depends on how much data is being stored in it.
(This method is therefore said to have self-expire functionality.)
``EXPENSIVESTAT'' is ``FALSE'' for this method.
CNFS has its own configuration file,
cycbuff.conf(5).
Storage method entries for the ``cnfs'' storage method must have an
``options'' field specifying the metacycbuff into which articles matching
that entry should be stored. See
cycbuff.conf(5)
for details on metacycbuffs.
timecaf
This method stores multiple articles in one file, whose name is based on
the article's arrival time and the storage class. The file name will be
<patharticles in inn.conf>/timecaf-nn/bb/aacc.CF,
where ``nn'' is the hexadecimal value of <storage_class> and ``bb'' and
``aacc'' are hexadecimal components of the arrival time. (The arrival
time in seconds since epoch is converted to hex and interpreted as
0xaabbccFF, with ``aa'', ``bb'', and ``cc'' used to build the path.) This
method does not have self-expire functionality (meaning
expire(8)
has to run periodically to delete old articles).
``EXPENSIVESTAT'' is ``FALSE'' for this method.
timehash
This method is very similar to ``timecaf'' except that each article is
stored in a separate file. The name of the file for a given article will
be
<patharticles in inn.conf>/time-nn/bb/cc/yyyy-aadd,
where ``nn'' is the hexadecimal value of <storage_class>, ``yyyy'' is a
hexadecimal sequence number, and ``bb'', ``cc'', and ``aadd'' are
components of the arrival time in hexadecimal (the arrival time is
converted to hex and interpreted as 0xaabbccdd). This method does not
have self-expire functionality.
``EXPENSIVESTAT'' is ``TRUE'' for this method.
tradspool
Traditional spool, or ``tradspool'', is the traditional news article
storage format. Each article is stored in a file named:
<patharticles in inn.conf>/news/group/name/nnnnn,
where ``news/group/name'' is the name of the newsgroup to which the
article was posted with each period changed to a slash, and ``nnnnn'' is
the sequence number of the article in that newsgroup. For crossposted
articles, the article is linked into each newsgroup to which it is
crossposted (using either hard or symbolic links). This is the way all
versions of INN prior to 2.0 stored all articles, as well as being the
article storage format used by C News and earlier news systems.
This method does not have self-expire functionality.
``EXPENSIVESTAT'' is ``TRUE'' for this method.
trash
This method silently discards all articles stored in it. Its only real
uses are for testing and for silently discarding articles matching a
particular storage method entry (for whatever reason). Articles stored in
this method take up no disk space and can never be retrieved, so this
method has self-expire functionality of a sort.
``EXPENSIVESTAT'' is ``FALSE'' for this method.
EXAMPLE
The following sample storage.conf file would store all articles posted to
alt.binaries.* in the ``BINARIES'' CNFS metacycbuff, all articles over
roughly 50KB in any other hierarchy in the ``LARGE'' CNFS metacycbuff, all
other articles in alt.* in one timehash class, and all other articles in
any newsgroups in a second timehash class, except for the internal.*
hierarchy which is stored in traditional spool format.
Notice that the last storage method entry will catch everything. This is
a good habit to get into; make sure that you have at least one catch-all
entry just in case something you didn't expect falls through the cracks.
Notice also that the special rule for the internal.* hierarchy is first,
so it will catch even articles crossposted to alt.binaries.* or over 50KB
in size.
HISTORY
Written by Katsuhiro Kondou <kondou@nec.co.jp> for InterNetNews.
This is revision 1.7.2.2, dated 2001/04/17.
