_I propose putting discussion posts somewhere in the vincity of
the secttion Individual reStructuredText Issues_
+## Design ##
+
**Goal**
To be able to use rst as a first-class markup language in ikiwiki. I think
most believe this is almost impossible (ikiwiki is built around markdown).
-**Design**
+## Wikilinks ##
**WikiLinks**, first and foremost, are needed for a wiki. rST already allows
specifying absolue and relative URL links, and relative links can be used to
**Proposal 2**.
This is a simple wiki page, with :wiki:`WikiLinks` and other_ links
-
+
.. _other: wiki:wikilink
+ We can get rid of the role part as well for WikiLinks::
+
+ .. default-role:: wiki
+
+ Enables `WikiLinks` but does not impact references such as ``other``
+ This can be made the default for ikiwiki.
+
Benefits of using a `:role:` and a `wiki: page/subpage` URL scheme are
following:
.. role:: wiki (title)
-*Implementation* there is no implementation of Proposal 2 but it should be
-doable; adding a local role is trivial. Rewriting `wiki:` links could be
-done in the format phase(?).
+### Implementation ###
+
+Implementation of Proposal-2 wikilinks are in the branch
+[rst-wikilinks][rst-wl]
+
+
+ This is a simple wiki page, with :wiki:`WikiLinks` and |named| links
+
+ .. |named| wiki:: Some Page
+
+ We can get rid of the role part as well for WikiLinks::
+
+ .. default-role:: wiki
+
+ Enables `WikiLinks` but does not impact references such as ``named``
+ This can be made the default for ikiwiki.
+
+[rst-wl]: http://github.com/engla/ikiwiki/commits/rst-wikilinks
+
+**rst-wikilinks** patch series includes changes at the end to use ikiwiki's
+'htmllink' for the links (which is the only sane thing to do to work in all configurations).
+This means a :wiki:`Link` should render just exactly like [[Link]] whether
+the target exists or not.
+
+On top of **rst-wikilinks** is [rst-customize][rst-custom] which adds two
+power user features: Global (python) file to read in custom directives
+(unsafe), and a wikifile as "header" file for all parsed .rst files (safe,
+but disruptive since all .rst depend on it). Well, the customizations have
+to be picked and chosen from this, but at least the global python file can
+be very convenient.
+
+> Did you consider just including the global rst header text into an item
+> in the setup file? --[[Joey]]
+>
+>> Then `rst_header` would not be much different from the python script
+>> `rst_customize`. rst_header is as safe as other files (though disruptive
+>> as noted), so it should/could be a editable file in the Wiki. A Python
+>> script of course can not be. There is nothing you can do in the
+>> rst_header (that you sensibly would do, I think) that couldn't be done in
+>> the Python script. `rst_header` has very limited use, but it is another
+>> possibility, mainly for the user-editable aspect. --[[ulrik]]
+>>
+>> (I foresaw only two things to be added to the rst_header: the default
+>> role could be configured there (as with rst_wikirole), and if you have a
+>> meta-role like :shortcut:, shortcuts could be defined there.)
+>
+> I have some discussion on the [docutils mailing list][dml], the developers
+> of docutils seems to favor "Proposal 1", while I defend my ideas. They
+> want all users of ReST to use only the basic featureset to remain
+> compatible, of course. -- [[ulrik]]
+
+[dml]: http://thread.gmane.org/gmane.text.docutils.user/5376
+
+Some rst-custom [examples are here](http://kaizer.se/wiki/rst_examples/)
+
+[rst-custom]: http://github.com/engla/ikiwiki/commits/rst-customize
+
+## Directives ##
Now **Directives**: As it is now, ikiwiki goes though (roughly):
filter, preprocess, htmlize, format as major stages of content
but rST directives allow a direct line (after :: on first line),
an option list, and a content block.
-
-**Discussion**
+> You've done a lot of work already, but ...
+>
+> The filter approach seems much simpler than the other approaches
+> for users to understand, since they can just use identical ikiwiki
+> markup on rst pages as they would use anywhere else. This is very desirable
+> if the wiki allows rst in addition to mdwn, since then users don't have
+> to learn two completly different ways of doing wikilinks and directives.
+> I also wonder if even those familiar with rst would find entirely natural
+> the ways you've found to shoehorn in wikilinks, named wikilinks, and ikiwiki
+> directives?
+>
+> Htmlize in filter avoids these problems. It also leaves open the possibility
+> that ikiwiki could become smarter about the rendering chain later, and learn
+> to use a better order for rst (ie, htmlize first). If that later happened,
+> the htmlize in filter hack could go away. --[[Joey]]
+
+> (BTW, the [[plugins/txt]] plugin already does html formatting
+> in filter, for similar reasons.) --[[Joey]]
+
+>> Thank you for the comments! Forget the work, it's not so much.
+>> I'd rank the :wiki: link addition pretty high, and the other changes way
+>> behind that:
+>>
+>> The :wiki:`Wiki Link` syntax is *very* appropriate as rst syntax
+>> since it fits well with other uses of roles (notice that :RFC:`822`
+>> inserts a link to RFC822 etc, and that the default role is a *title* role
+>> (title of some work); thus very appropriate for medium-specific links like
+>> wiki links. So I'd rank :wiki: links a worthwhile addition regardless of
+>> outcome here, since it's a very rst-like alternative for those who wish to
+>> use more rst-like syntax (and documents degrades better outside the wiki as
+>> noted).
+>>
+>>> Unsure about the degredation argument. It will work some of
+>>> the time, but ikiwiki's [[ikiwiki/subpage/linkingrules]]
+>>> are sufficiently different from normal html relative link
+>>> rules that it often won't work. --[[Joey]]
+>>>
+>>>> With degradation I mean that if you take a file out of the wiki; the
+>>>> links degrade to stylized text. If using default role, they degrade to
+>>>> :title: which renders italicized text (which I find is exactly
+>>>> appropriate). There is no way for them to degrade into links, except of
+>>>> course if you reimplement the :wiki: role. You can also respecify
+>>>> either the default role (the `wikilink` syntax) or the :wiki: role (the
+>>>> :wiki:`wikilink` syntax) to any other markup, for example None.
+>>>> --[[ulrik]]
+>>
+>> The named link syntax (just like the :wiki: role) are inspired from
+>> [trac][tracrst] and a good fit, but only if the wiki is committed to
+>> using only rst, which I don't think is the case.
+>>
+>> The rst-customize changes are very useful for custom directive
+>> installations (like the sourcecode directive, or shortcut roles I show
+>> in the examples page), but there might be a way for the user to inject
+>> docutils addons that I'm missing (one very ugly way would be to stick
+>> them in sitecustomize.py which affects all Python programs).
+>>
+>> With the presented changes, I already have a working RestructuredText
+>> wiki, but I'm admitting that using .. raw:: html around all directives is
+>> very ugly (I use few directives: inline, toggle, meta, tag, map)
+>>
+>> On filter/htmlize: Well **rst** is clearly antisocial: It can't see HTML,
+>> and ikiwiki directives are wrappend in paragraph tags. (For wikilinks
+>> this is probably no problem). So the suggestion about `.. ikiwiki:` is
+>> partly because it looks good in rst syntax, but also since it would emit
+>> a div to wrap around the element instead of a paragraph.
+>>
+>> I don't know if you mean that rst could be reordered to do htmlize before
+>> other phases? rst must be before any preprocess hook to avoid seeing any
+>> HTML.
+>>
+>>> One of my long term goals is to refactor all the code in ikiwiki
+>>> that manually runs the various stages of the render pipeline,
+>>> into one centralized place. Once that's done, that place can get
+>>> smart about what order to run the stages, and use a different
+>>> order for rst. --[[Joey]]
+>>
+>> If I'm thinking right, processing to HTML already in filter means any
+>> processing in scan can be reused directly (or skipped if it's legal to
+>> emit 'add_link' in filter.)
+>>
+>> -- [[ulrik]]
+
+>>> Seems it could be, yes. --[[Joey]]
+>>>
+>>>> It is not clear how we can work around reST wrapping directives with
+>>>> paragraph tags. Also, some escaping of xml characters & <> might
+>>>> happen, but I can't imagine right now what breakage can come from that.
+>>>> -- [[ulrik]]
+
+[tracrst]: http://trac.edgewall.org/wiki/WikiRestructuredText
+
+### Implementation ###
+
+Preserving indents in the preprocessor are in branch [pproc-indent][ppi]
+
+(These simple patches come with a warning: _Those are the first lines of
+Perl I've ever written!_)
+
+> This seems like a good idea, since it solves issues for eg, indented
+> directives in mdwn as well. But, looking at the diff, I see a clear bug:
+>
+> - return "[[!$command <span class=\"error\">".
+> + $result = "[[!$command <span class=\"error\">".
+>
+> That makes it go on and parse an infinitely nested directive chain, instead
+> of immediatly throwing an error.
+>
+> Also, it seems that the "indent" matching in the regexps may be too broad,
+> wouldn't it also match whitespace before a directive that was not at the beginning
+> of a line, and treat it as an indent? With some bad luck, that could cause mdwn
+> to put the indented output in a pre block. --[[Joey]]
+>
+>> You are probably right about the bug. I'm not quite sure what the nested
+>> directives examples looks like, but I must have overlooked how the
+>> recursion counter works; I thought simply changing if to elif the next
+>> few lines would solve that. I'm sorry for that!
+>>
+>> We don't have to change the `$handle` function at all, if it is possible
+>> to do the indent substitution all in one line instead of passing it to
+>> handle, I don't know if it is possible to turn:
+>>
+>> $content =~ s{$regex}{$handle->($1, $2, $3, $4, $5)}eg;
+>>
+>> into
+>>
+>> $content =~ s{$regex}{s/^/$1/gm{$handle->($2, $3, $4, $5)}}eg;
+>>
+>> Well, no idea how that would be expressed, but I mean, replace the indent
+>> directly in $handle's return value.
+>>
+>>> Yes, in effect just `indent($1, handle->($2,$,4))` --[[Joey]]
+>>
+>> The indent-catching regex is wrong in the way you mention, it has been
+>> nagigng my mind a bit as well; I think matching start of line + spaces
+>> and tabs is the only thing we want.
+>> -- [[ulrik]]
+>>
+>>> Well, seems you want to match the indent at the start of the line containing
+>>> the directive, even if the directive does not start the line. That would
+>>> be quite hard to make a regexp do, though. --[[Joey]]
+>>
+>> I wasted a long time getting the simpler `indent($1, handle->($2,$,4))` to
+>> work (remember, I don't know perl at all). Somehow `$1` does not arrive, I
+>> made a simple testcase that worked, and I conclude something inside $handle
+>> results in the value of $1 not arriving as it should!
+>>
+>> Anyway, instead a very simple incremental patch is in [pproc-indent][ppi]
+>> where the indentation regex is `(^[ \t]+|)` instead, which seems to work
+>> very well (and the regex is multiline now as well). I'm happy to rebase the
+>> changes if you want or you can just squash the four patches 1+3 => 1+1
+>> -- [[ulrik]]
+
+[ppi]: http://github.com/engla/ikiwiki/commits/pproc-indent
+
+## Discussion ##
I guess you (or someone) has been through this before and knows why it
simply won't work. But I hoped there was something original in the above;
* We resolve rST links without definition, we don't help resolving defined
relative links, so we don't support specifying link name and target
separately.
-
-> I found out this is possible by using rST subsitutions. So to do [[Version history...|releases]]
-> you would use:
->
-> `|releases|_`
-> `.. |releases| replace:: Version history...`
-> Which does not seem to have an inline equivalent. Using non-resolved links there is the alternative:
->
-> ``Version history <releases/>`_`. --ulrik [kaizer.se]
+
+ * Resolved by |replacement| links with the wiki:: directive.
**A first implementation: Resolving unmatched links**
>> However, I think that if the cache does not work for a big load, it should
>> not work at all; small loads are small so they don't matter. --ulrik
-Patch follows:
-
-----
-<pre>
- From 486fd79e520da1d462f00f40e7a90ab07e9c6fdf Mon Sep 17 00:00:00 2001
- From: Ulrik Sverdrup <ulrik.sverdrup@gmail.com>
- Date: Thu, 17 Sep 2009 15:18:50 +0200
- Subject: [PATCH] rst: Resolve native reStructuredText links to ikiwiki pages
-
- Links in rST use syntax `Like This`_ or OneWordLink_, and are
- generally used for relative or absolue links, with an auxiliary
- definition:
-
- .. _`Like This`: http://ikiwiki.info
- .. _OneWordLink: relative
-
- We can hook into docutils to resolve unresolved links so that rST
- links without definition can be resolved to wiki pages. This enables
- WikiLink_ to link to [[WikiLink]] (if no .. _WikiLink is specified).
-
- Comparing to Ikiwiki's wikilinks
-
- [[blogging|blog]] specifies a link to the page blog, with the name
- blogging. In rST we should use blogging_
-
- .. _blogging: blog
-
- *However*, note that this patch does not hook into this. What we
- resolve in this patch is finding the appropriate "_blogging" if it is
- not found, not resolving the address 'blog'.
- ---
- plugins/rst | 46 +++++++++++++++++++++++++++++++++++++++++-----
- 1 files changed, 41 insertions(+), 5 deletions(-)
-
- diff --git a/plugins/rst b/plugins/rst
- index a2d07eb..a74baa8 100755
- --- a/plugins/rst
- +++ b/plugins/rst
- @@ -6,22 +6,58 @@
- # based a little bit on rst.pm by Sergio Talens-Oliag, but only a little bit. :)
- #
- # Copyright © martin f. krafft <madduck@madduck.net>
- +# Copyright © Ulrik Sverdrup <ulrik.sverdrup@gmail.com>, 2009
- +#
- # Released under the terms of the GNU GPL version 2
- #
- +
- __name__ = 'rst'
- __description__ = 'xml-rpc-based ikiwiki plugin to process RST files'
- -__version__ = '0.3'
- +__version__ = '0.3+'
- __author__ = 'martin f. krafft <madduck@madduck.net>'
- __copyright__ = 'Copyright © ' + __author__
- __licence__ = 'GPLv2'
-
- from docutils.core import publish_parts;
- +from docutils.writers import html4css1
- from proxy import IkiWikiProcedureProxy
-
- -def rst2html(proxy, *kwargs):
- - # FIXME arguments should be treated as a hash, the order could change
- - # at any time and break this.
- - parts = publish_parts(kwargs[3], writer_name='html',
- +class IkiwikiWriter(html4css1.Writer):
- + def resolve_node(self, node):
- + refname = node.get('refname', None)
- + if not refname:
- + return False
- +
- + bestlink = self.proxy.rpc('bestlink', self.page, refname)
- +
- + node.resolved = 1
- + node['class'] = 'wiki'
- + del node['refname']
- +
- + if not bestlink:
- + rel_url = "#"
- + else:
- + rel_url = self.proxy.rpc('urlto', bestlink, self.page)
- + self.proxy.rpc('add_link', self.page, bestlink)
- + node['refuri'] = rel_url
- + self.proxy.rpc('debug', "Emitting link %s => %s" % (refname, rel_url))
- + return True
- +
- + resolve_node.priority = 1
- +
- + def __init__(self, proxy, page):
- + html4css1.Writer.__init__(self)
- + self.proxy = proxy
- + self.page = page
- + self.unknown_reference_resolvers = (self.resolve_node, )
- +
- +def rst2html(proxy, *args):
- + # args is a list paired by key, value, so we turn it into a dict
- + kwargs = dict((k, v) for k, v in zip(*[iter(args)]*2))
- + page = kwargs['page']
- +
- + parts = publish_parts(kwargs['content'],
- + writer=IkiwikiWriter(proxy, page),
- settings_overrides = { 'halt_level': 6
- , 'file_insertion_enabled': 0
- , 'raw_enabled': 1
- --
- 1.6.4
-
-</pre>
-----