If [[tips/untrusted_git_push]] is setup, one can edit the PO files in one's
preferred `$EDITOR`, without needing to be online.
+Markup languages support
+------------------------
+
+Markdown is well supported. Some other markup languages supported by
+ikiwiki mostly work, but some pieces of syntax are not rendered
+correctly on the slave pages:
+
+* [[reStructuredText|rst]]: anonymous hyperlinks and internal
+ cross-references
+* [[wikitext]]: conversion of newlines to paragraphs
+* [[creole]]: verbatim text is wrapped, tables are broken
+* [[html]] and LaTeX: not supported yet; the dedicated po4a modules
+ could be used to support them, but they would need a security audit
+* other markup languages have not been tested.
+
+
TODO
====
read any file. NB: this hack depends on po4a internals to stay
the same.
-#### To be checked
-
##### Locale::Po4a modules
The modules we want to use have to be checked, as not all are safe
> Freaky code, but seems ok due to use of `quotementa`.
+#### To be checked
+
##### Text::WrapI18N
`Text::WrapI18N` can cause DoS (see the
[Debian bug #470250](http://bugs.debian.org/470250)), but it is
optional and we do not need the features it provides.
-It is loaded if available by `Locale::Po4a::Common`; looking at the
-code, I'm not sure we can prevent this at all, but maybe some symbol
-table manipulation tricks could work; overriding
-`Locale::Po4a::Common::wrapi18n` may be easier. I'm no expert at all
-in this field. Joey? [[--intrigeri]]
-
-> Update: Nicolas François suggests we add an option to po4a to
-> disable it. It would do the trick, but only for people running
-> a brand new po4a (probably too late for Lenny). Anyway, this option
-> would have to take effect in a `BEGIN` / `eval` that I'm not
-> familiar with. I can learn and do it, in case no Perl wizard
-> volunteers to provide the po4a patch. [[--intrigeri]]
-
->> That doesn't really need to be in a BEGIN. This patch moves it to
->> `import`, and makes this disable wrap18n:
->> `use Locale::Po4a::Common q{nowrapi18n}` --[[Joey]]
-
-<pre>
---- /usr/share/perl5/Locale/Po4a/Common.pm 2008-07-21 14:54:52.000000000 -0400
-+++ Common.pm 2008-11-11 18:27:34.000000000 -0500
-@@ -30,8 +30,16 @@
- use strict;
- use warnings;
-
--BEGIN {
-- if (eval { require Text::WrapI18N }) {
-+sub import {
-+ my $class=shift;
-+ my $wrapi18n=1;
-+ if ($_[0] eq 'nowrapi18n') {
-+ shift;
-+ $wrapi18n=0;
-+ }
-+ $class->export_to_level(1, $class, @_);
-+
-+ if ($wrapi18n && eval { require Text::WrapI18N }) {
-
- # Don't bother determining the wrap column if we cannot wrap.
- my $col=$ENV{COLUMNS};
-</pre>
+> I proposed a patch based on Joey's to po4a-devel, allowing to fully
+> disable this module's use. When it is merged upstream, we'll need to add
+> `use Locale::Po4a::Common qw(nowrapi18n)` to `po.pm`, before loading
+> any other `Locale::Po4a` module. A versioned dependency may be needed.
+> --[[intrigeri]]
##### Term::ReadKey
`libterm-readkey-perl`, so it will probably be installed on most
systems using the po plugin.
-If `$ENV{COLUMNS}` is not set, `Locale::Po4a::Common` uses
-`Term::ReadKey::GetTerminalSize()` to get the terminal size. How safe
-is this?
+`Term::ReadKey` has too far reaching implications for us to
+be able to guarantee anything wrt. security.
-Part of `Term::ReadKey` is written in C. Depending on the runtime
-platform, this function use ioctl, environment, or C library function
-calls, and may end up running the `resize` command (without
-arguments).
+> The option that disables `Text::WrapI18N` also disables
+> `Term::ReadKey` as a consequence. [[--intrigeri]]
-IMHO, using Term::ReadKey has too far reaching implications for us to
-be able to guarantee anything wrt. security. Since it is anyway of no
-use in our case, I suggest we define `ENV{COLUMNS}` before loading
-`Locale::Po4a::Common`, just to be on the safe side. Joey?
-[[--intrigeri]]
+### msgmerge
-> Update: adding an option to disable `Text::WrapI18N`, as Nicolas
-> François suggested, would as a bonus disable `Term::ReadKey`
-> as well. [[--intrigeri]]
+`refreshpofiles()` runs this external program.
-### msgmerge
+A po4a developer answered he does "not expect any security issues from
+it". I did not manage to crash it with `zzuf`, nor was able to find
+any past security holes.
-`refreshpofiles()` runs this external program. A po4a developer
-answered he does "not expect any security issues from it".
+### msgfmt
-### Fuzzing input
+`isvalidpo()` runs this external program.
-I was not able to find any public information about gettext or po4a
-having been tested with a fuzzing program, such as `zzuf` or `fusil`.
-Moreover, some gettext parsers seem to be quite
-[easy to crash](http://fusil.hachoir.org/trac/browser/trunk/fuzzers/fusil-gettext),
-so it might be useful to bang msgmerge/po4a's heads against such
-a program in order to easily detect some of the most obvious DoS.
-[[--intrigeri]]
+* I could not manage to make it behave badly using zzuf, it exits
+ cleanly when too many errors are detected.
+* I could not find any past security holes.
-> po4a was not fuzzy-tested, but according to one of its developers,
-> "it would be really appreciated". [[--intrigeri]]
+### Fuzzing input
Test conditions:
... seems to lose the fight, at the `readpo(LICENSES.fr.po)` step,
against some kind of infinite loop, deadlock, or any similar beast.
-It does not seem to eat memory, though.
-
-Whatever format module is used does not change anything. This is thus
-probably a bug in po4a's core or in a lib it depends on.
-
-The sub `read`, in `TransTractor.pm`, seems to be a good debugging
-starting point.
-#### msgmerge
-
-`msgmerge` is run in our `refreshpofiles` function. I did not manage
-to crash it with `zzuf`.
+The root of this bug lies in `Text::WrapI18N`, see above for
+possible solutions.
gettext/po4a rough corners
--------------------------
### Page title in links
-To use the page titles set with the [[meta|plugins/meta]] plugin when
-rendering links would be very much nicer, than the current
-"filename.LL" format. This is actually a duplicate for
-[[bugs/pagetitle_function_does_not_respect_meta_titles]].
-
-Going to work on this in my `meta` branch.
-
-### Translation status in links
-
-See [[contrib/po]].
-
-### Backlinks
-
-They are not updated when the source page changes (e.g. meta title).
-
-Page formats
-------------
-
-Markdown is well supported, great, but what about others?
-
-The [[po|plugins/po]] uses `Locale::Po4a::Text` for every page format;
-this can be expected to work out of the box with most other wiki-like
-formats supported by ikiwiki. Some of their ad-hoc syntax might be
-parsed in a strange way, but the worst problems I can imagine would be
-wrapping issues; e.g. there is code in po4a dedicated to prevent
-re-wrapping the underlined Markdown headers.
-
-While it would be easy to better support formats such as [[html]] or
-LaTeX, by using for each one the dedicated po4a module, this can be
-problematic from a security point of view.
-
-**TODO**: test the more popular formats and write proper documentation
-about it.
-
-Translation quality assurance
------------------------------
-
-Modifying a PO file via the CGI must be forbidden if the new version
-is not a valid PO file. As a bonus, check that it provides a more
-complete translation than the existing one.
-
-A new `cansave` type of hook would be needed to implement this.
-
-Note: committing to the underlying repository is a way to bypass
-this check.
-
-Creating new pages on the web
------------------------------
-
-See [[contrib/po|contrib/po]].
-
-Renaming pages
---------------
-
-- Renaming a translation should be forbidden.
+Using the fix to
+[[bugs/pagetitle_function_does_not_respect_meta_titles]] from
+[[intrigeri]]'s `meta` branch, the generated links' text is based on
+the page titles set with the [[meta|plugins/meta]] plugin. This has to
+be merged into ikiwiki upstream, though.
Robustness tests
----------------
-### Disabling the plugin
+### Enabling/disabling the plugin
- enabling the plugin with `po_translatable_pages` set
- enabling the plugin without `po_translatable_pages` set: **OK**
`po_slave_languages`: needs two rebuilds, but **OK** (this is quite
a perverse test actually)
-### Creating pages
-
-- creating a master page via RCS: **OK**
-- creating a master page via CGI: **OK**
-
-### Deleting pages
-
-- removing a master page via RCS: **OK**
-- removing a translation via RCS: **OK**
-- removing a master page via CGI: **OK**
-- removing a translation via CGI: **OK**
-
-### Renaming pages
+### Creating/deleting/renaming pages
-- renaming a master page via RCS: **OK** (but the old translations
- are lost, because not all RCS track file renaming)
-- renaming a master page and its translations via RCS: **OK**
-- renaming a master page via CGI: **OK**
-- renaming a translation via RCS
-- renaming a translation via CGI
+All cases of master/slave page creation/deletion/rename, both via RCS
+and via CGI, have been tested.
### Misc