Document the 'generated' keyword argument

[git.ikiwiki.info.git] / doc / plugins / aggregate.mdwn

diff --git a/doc/plugins/aggregate.mdwn b/doc/plugins/aggregate.mdwn
index 4cd5b57acc4316f31a115e4a841b1ad66cf6ac2c..ecca0232ea0aacbac18a451dc3446a10d27787f5 100644 (file)
--- a/doc/plugins/aggregate.mdwn
+++ b/doc/plugins/aggregate.mdwn
@@ -1,19 +1,31 @@
-This plugin allows content from other blogs to be aggregated into the wiki.
-Aggregate a blog as follows:
+[[!template id=plugin name=aggregate author="[[Joey]]"]]
+[[!tag type/useful]]

 
 

-       \[[aggregate name="example blog" feedurl="http://example.com/index.rss" url="http://example.com/" updateinterval="15"]
+This plugin allows content from other feeds to be aggregated into the wiki.
+Aggregate a feed as follows:

 
 

-That example aggregates posts from the expecified RSS feed, updating no
+       \[[!aggregate name="example blog" dir="example"
+       feedurl="http://example.com/index.rss"
+       url="http://example.com/" updateinterval="15"]]
+
+That example aggregates posts from the specified RSS feed, updating no

 more frequently than once every 15 minutes, and puts a page per post under
 the example/ directory in the wiki.
 
 more frequently than once every 15 minutes, and puts a page per post under
 the example/ directory in the wiki.
 

-You can then use ikiwiki's [[blog]] support to create a blog of one or more
-aggregated feeds.
+You can then use ikiwiki's [[ikiwiki/blog]] support to create a blog of one or
+more aggregated feeds. For example:
+
+       \[[!inline pages="internal(example/*)"]]

 
 ## setup
 
 
 ## setup
 

-Make sure that you have the [[html]] plugin enabled, as the created pages are
-in html format. The [[meta]] and [[tag]] plugins are also recommended.
+New users of aggregate should enable the `aggregateinternal => 1` option in the
+.setup file. If you don't do so, you will need to enable the [[html]] plugin
+as well as aggregate itself, since feed entries will be stored as HTML.
+
+The [[meta]] and [[tag]] plugins are also recommended. The
+[[htmltidy]] plugin is suggested, since feeds can easily contain html
+problems, some of which tidy can fix.

 
 You will need to run ikiwiki periodically from a cron job, passing it the
 --aggregate parameter, to make it check for new posts. Here's an example
 
 You will need to run ikiwiki periodically from a cron job, passing it the
 --aggregate parameter, to make it check for new posts. Here's an example


@@ -21,6 +33,14 @@ crontab entry:
 
        */15 * * * * ikiwiki --setup my.wiki --aggregate --refresh
 
 
        */15 * * * * ikiwiki --setup my.wiki --aggregate --refresh
 

+Alternatively, you can allow `ikiwiki.cgi` to trigger the aggregation. You
+should only need this if for some reason you cannot use cron, and instead
+want to use a service such as [WebCron](http://webcron.org). To enable
+this, turn on `aggregate_webtrigger` in your setup file. The url to
+visit is `http://whatever/ikiwiki.cgi?do=aggregate_webtrigger`. Anyone
+can visit the url to trigger an aggregation run, but it will only check
+each feed if its `updateinterval` has passed.
+

 ## usage
 
 Here are descriptions of all the supported parameters to the `aggregate`
 ## usage
 
 Here are descriptions of all the supported parameters to the `aggregate`


@@ -28,7 +48,7 @@ directive:
 
 * `name` - A name for the feed. Each feed must have a unique name.
   Required.
 
 * `name` - A name for the feed. Each feed must have a unique name.
   Required.

-* `url` - The url to the web page for the blog that's being aggregated.
+* `url` - The url to the web page for the feed that's being aggregated.

   Required.
 * `dir` - The directory in the wiki where pages should be saved. Optional,
   if not specified, the directory is based on the name of the feed.
   Required.
 * `dir` - The directory in the wiki where pages should be saved. Optional,
   if not specified, the directory is based on the name of the feed.


@@ -36,19 +56,52 @@ directive:
   will look for feeds on the `url`. RSS and atom feeds are supported.
 * `updateinterval` - How often to check for new posts, in minutes. Default
   is 15 minutes.
   will look for feeds on the `url`. RSS and atom feeds are supported.
 * `updateinterval` - How often to check for new posts, in minutes. Default
   is 15 minutes.

-* `expireage` - Expire old items from this blog if they are older than
+* `expireage` - Expire old items from this feed if they are older than

   a specified number of days. Default is to never expire on age.
   a specified number of days. Default is to never expire on age.

-* `expirecount` - Expire old items from this blog if there are more than
+* `expirecount` - Expire old items from this feed if there are more than

   the specified number total. Oldest items will be expired first. Default
   is to never expire on count.
   the specified number total. Oldest items will be expired first. Default
   is to never expire on count.

-* `tag` - A tag to tag each post from the blog with. A good tag to use is
-  the name of the blog. Can be repeated multiple times. The [[tag]] plugin
+* `tag` - A tag to tag each post from the feed with. A good tag to use is
+  the name of the feed. Can be repeated multiple times. The [[tag]] plugin

   must be enabled for this to work.
   must be enabled for this to work.

+* `template` - Template to use for creating the aggregated pages. Defaults to
+  aggregatepost.

 
 Note that even if you are using subversion or another revision control
 system, pages created by aggregation will *not* be checked into revision
 control.
 
 
 Note that even if you are using subversion or another revision control
 system, pages created by aggregation will *not* be checked into revision
 control.
 

-This plugin is not enabled by default.
+## internal pages and `aggregateinternal`
+
+This plugin creates a page for each aggregated item. 
+
+If the `aggregateinternal` option is enabled in the setup file (which is
+recommended), aggregated pages are stored in the source directory with a
+"._aggregated" extension. These pages cannot be edited by web users, and
+do not generate first-class wiki pages. They can still be inlined into a
+blog, but you have to use `internal` in [[PageSpecs|IkiWiki/PageSpec]],
+like `internal(blog/*)`.
+
+For backward compatibility, the default is that these pages have the
+".html" extension, and are first-class wiki pages -- each one generates
+a separate HTML page in the output, and they can even be edited.
+
+That turns out to not be ideal for aggregated content, because publishing
+files for each of those pages is a waste of disk space and CPU, and you
+probably don't want to allow them to be edited. So, there is an alternative
+method that can be used (and is recommended), turned on by the
+`aggregateinternal` option in the setup file.
+
+If you are already using aggregate and want to enable `aggregateinternal`,
+you should follow this process:

 
 

-[[tag type/useful]]
+1. Update all [[PageSpecs|ikiwiki/PageSpec]] that refer to the aggregated
+   pages -- such as those in inlines. Put "internal()" around globs 
+   in those PageSpecs. For example, if the PageSpec was `foo/*`, it should
+   be changed to `internal(foo/*)`. This has to be done because internal
+   pages are not matched by regular globs.
+2. Turn on `aggregateinternal` in the setup file.
+3. Use [[ikiwiki-transition]] to rename all existing aggregated `.html`
+   files in the srcdir. The command to run is
+   `ikiwiki-transition aggregateinternal $setupfile`,
+4. Refresh the wiki. (`ikiwiki -setup your.setup -refresh`)