-I have a working minimal implementation letting the rst renderer resolve undefined native rST links to ikiwiki pages. I have posted it as one patch at:
+_NB! this page has been refactored, hopefully it is clearer now_
+_I propose putting discussion posts somewhere in the vincity of
+the secttion Individual reStructuredText Issues_
+
+## Design ##
+
+**Goal**
+
+To be able to use rst as a first-class markup language in ikiwiki. I think
+most believe this is almost impossible (ikiwiki is built around markdown).
+
+## Wikilinks ##
+
+**WikiLinks**, first and foremost, are needed for a wiki. rST already allows
+specifying absolue and relative URL links, and relative links can be used to
+tie together wiki of rst documents.
+
+1. Below are links to a small, working implementation for resolving
+ undefined rST references using ikiwiki's mechanism. This is **Proposal 1**
+ for rst WikiLinks.
+
+2. Looking over at rST-using systems such as trac and MoinMoin; I think it
+ would be wiser to implement wikilinks by the `:role:` mechanism, together
+ with allowing a custom URL scheme to point to wiki links. This is
+ **Proposal 2**.
+
+ This is a simple wiki page, with :wiki:`WikiLinks` and other_ links
+
+ .. _other: wiki:wikilink
+
+ We can get rid of the role part as well for WikiLinks::
+
+ .. default-role:: wiki
+
+ Enables `WikiLinks` but does not impact references such as ``other``
+ This can be made the default for ikiwiki.
+
+Benefits of using a `:role:` and a `wiki: page/subpage` URL scheme are
+following:
+
+1. rST documents taken out of the context (the wiki) will not fail as bad as
+ if they have lots of Proposal-1 links: They look just the same as valid
+ references, and you have to edit them all.
+ In contrast, should the `:wiki:` role disappear, one line is enough
+ to redefined it and silence all the warnings for the document:
+
+ .. role:: wiki (title)
+
+### Implementation ###
+
+Implementation of Proposal-2 wikilinks are in the branch
+[rst-wikilinks][rst-wl]
+
+
+ This is a simple wiki page, with :wiki:`WikiLinks` and |named| links
+
+ .. |named| wiki:: Some Page
+
+ We can get rid of the role part as well for WikiLinks::
+
+ .. default-role:: wiki
+
+ Enables `WikiLinks` but does not impact references such as ``named``
+ This can be made the default for ikiwiki.
+
+[rst-wl]: http://github.com/engla/ikiwiki/commits/rst-wikilinks
+
+**rst-wikilinks** patch series includes changes at the end to use ikiwiki's
+'htmllink' for the links (which is the only sane thing to do to work in all configurations).
+This means a :wiki:`Link` should render just exactly like [[Link]] whether
+the target exists or not.
+
+On top of **rst-wikilinks** is [rst-customize][rst-custom] which adds two
+power user features: Global (python) file to read in custom directives
+(unsafe), and a wikifile as "header" file for all parsed .rst files (safe,
+but disruptive since all .rst depend on it). Well, the customizations have
+to be picked and chosen from this, but at least the global python file can
+be very convenient.
+
+Some rst-custom [examples are here](http://kaizer.se/wiki/rst_examples/)
+
+[rst-custom]: http://github.com/engla/ikiwiki/commits/rst-customize
+
+## Directives ##
+
+Now **Directives**: As it is now, ikiwiki goes though (roughly):
+filter, preprocess, htmlize, format as major stages of content
+transformation. rST has major problems to work with any HTML that enters the
+picture before it.
+
+1. Formatting rST in `htmlize` (as is done now): Raw html can be escaped by
+ raw blocks:
+
+ .. raw:: html
+
+ \[[!inline and do stuff]]
+
+ (This can be simplified to alias the above as `.. ikiwiki::`)
+ This escape method works, if ikwiki can be persuaded to maintain the
+ indent when inserting html, so that it stays inside the raw block.
+
+2. Formatting rST in `filter` (idea)
+ 1. rST does not have to see any HTML (raw not needed)
+ 2. rST directives can alias ikiwiki syntax:
+
+ ..ikiwiki:: inline pages= ...
+
+ 3. Using rST directives as ikiwiki directives can be complicated;
+ but rST directives allow a direct line (after :: on first line),
+ an option list, and a content block.
+
+### Implementation ###
+
+Preserving indents in the preprocessor are in branch [pproc-indent][ppi]
+
+(These simple patches come with a warning: _Those are the first lines of
+Perl I've ever written!_)
+
+[ppi]: http://github.com/engla/ikiwiki/commits/pproc-indent
+
+## Discussion ##
+
+I guess you (or someone) has been through this before and knows why it
+simply won't work. But I hoped there was something original in the above;
+and I know there are wiki installations where rST works. --ulrik
+
+**Individual reStructuredText Issues**
+
+* We resolve rST links without definition, we don't help resolving defined
+ relative links, so we don't support specifying link name and target
+ separately.
+
+ * Resolved by |replacement| links with the wiki:: directive.
+
+**A first implementation: Resolving unmatched links**
+
+I have a working minimal implementation letting the rst renderer resolve
+undefined native rST links to ikiwiki pages. I have posted it as one patch at:
Preview commit: http://github.com/engla/ikiwiki/commit/486fd79e520da1d462f00f40e7a90ab07e9c6fdf
Repository: git://github.com/engla/ikiwiki.git
Design issues of the patch:
-Right now it changes rendering so that undefined pages (previous errors) are resolved to either ikiwiki pages or link to "#". It could be changed (trivially) so that undefined pages give the same error as before. Since it only resolves links that would previously error out, impact on current installations should be minimal.
-
-I don't know why backlinks don't show up with the patch as it is; they are registered, but not rendered on the linked-to page.
-
-Desing issues in general:
-
-We resolve rST links without definition, we don't help resolving defined relative links, so we don't support specifying link name and target separately.
-
-Many other issues with rST are of course unresolved, but some might be solved by implementing custom rST directives (which is a supported extension mechanism).
-
-Patch follows:
-
-----
-<pre>
- From 486fd79e520da1d462f00f40e7a90ab07e9c6fdf Mon Sep 17 00:00:00 2001
- From: Ulrik Sverdrup <ulrik.sverdrup@gmail.com>
- Date: Thu, 17 Sep 2009 15:18:50 +0200
- Subject: [PATCH] rst: Resolve native reStructuredText links to ikiwiki pages
-
- Links in rST use syntax `Like This`_ or OneWordLink_, and are
- generally used for relative or absolue links, with an auxiliary
- definition:
-
- .. _`Like This`: http://ikiwiki.info
- .. _OneWordLink: relative
-
- We can hook into docutils to resolve unresolved links so that rST
- links without definition can be resolved to wiki pages. This enables
- WikiLink_ to link to [[WikiLink]] (if no .. _WikiLink is specified).
-
- Comparing to Ikiwiki's wikilinks
-
- [[blogging|blog]] specifies a link to the page blog, with the name
- blogging. In rST we should use blogging_
-
- .. _blogging: blog
-
- *However*, note that this patch does not hook into this. What we
- resolve in this patch is finding the appropriate "_blogging" if it is
- not found, not resolving the address 'blog'.
- ---
- plugins/rst | 46 +++++++++++++++++++++++++++++++++++++++++-----
- 1 files changed, 41 insertions(+), 5 deletions(-)
-
- diff --git a/plugins/rst b/plugins/rst
- index a2d07eb..a74baa8 100755
- --- a/plugins/rst
- +++ b/plugins/rst
- @@ -6,22 +6,58 @@
- # based a little bit on rst.pm by Sergio Talens-Oliag, but only a little bit. :)
- #
- # Copyright © martin f. krafft <madduck@madduck.net>
- +# Copyright © Ulrik Sverdrup <ulrik.sverdrup@gmail.com>, 2009
- +#
- # Released under the terms of the GNU GPL version 2
- #
- +
- __name__ = 'rst'
- __description__ = 'xml-rpc-based ikiwiki plugin to process RST files'
- -__version__ = '0.3'
- +__version__ = '0.3+'
- __author__ = 'martin f. krafft <madduck@madduck.net>'
- __copyright__ = 'Copyright © ' + __author__
- __licence__ = 'GPLv2'
-
- from docutils.core import publish_parts;
- +from docutils.writers import html4css1
- from proxy import IkiWikiProcedureProxy
-
- -def rst2html(proxy, *kwargs):
- - # FIXME arguments should be treated as a hash, the order could change
- - # at any time and break this.
- - parts = publish_parts(kwargs[3], writer_name='html',
- +class IkiwikiWriter(html4css1.Writer):
- + def resolve_node(self, node):
- + refname = node.get('refname', None)
- + if not refname:
- + return False
- +
- + bestlink = self.proxy.rpc('bestlink', self.page, refname)
- +
- + node.resolved = 1
- + node['class'] = 'wiki'
- + del node['refname']
- +
- + if not bestlink:
- + rel_url = "#"
- + else:
- + rel_url = self.proxy.rpc('urlto', bestlink, self.page)
- + self.proxy.rpc('add_link', self.page, bestlink)
- + node['refuri'] = rel_url
- + self.proxy.rpc('debug', "Emitting link %s => %s" % (refname, rel_url))
- + return True
- +
- + resolve_node.priority = 1
- +
- + def __init__(self, proxy, page):
- + html4css1.Writer.__init__(self)
- + self.proxy = proxy
- + self.page = page
- + self.unknown_reference_resolvers = (self.resolve_node, )
- +
- +def rst2html(proxy, *args):
- + # args is a list paired by key, value, so we turn it into a dict
- + kwargs = dict((k, v) for k, v in zip(*[iter(args)]*2))
- + page = kwargs['page']
- +
- + parts = publish_parts(kwargs['content'],
- + writer=IkiwikiWriter(proxy, page),
- settings_overrides = { 'halt_level': 6
- , 'file_insertion_enabled': 0
- , 'raw_enabled': 1
- --
- 1.6.4
-
-</pre>
-----
+The page is rST-parsed once in 'scan' and once in 'htmlize' (the first to generate backlinks). Can the parse output be safely reused?
+
+> The page content fed to htmlize may be different than that fed to scan,
+> as directives can change the content. If you cached the input and output
+> at scan time, you could reuse the cached data at htmlize time for inputs
+> that are the same -- but that could be a very big cache! --[[Joey]]
+
+>> I would propose using a simple heuristic: If you see \[[ anywhere on the
+>> page, don't cache it. It would be an effective cache for pure-rst wikis
+>> (without any ikiwiki directives or wikilinks).
+>> However, I think that if the cache does not work for a big load, it should
+>> not work at all; small loads are small so they don't matter. --ulrik
+