X-Git-Url: http://git.vanrenterghem.biz/git.ikiwiki.info.git/blobdiff_plain/eba2dee941c18b3a1bd9c8af52c01ba7b5b46a64..523718b181e9767d322bba3fe0b68cfcf85056a6:/doc/plugins/po.mdwn?ds=inline

diff --git a/doc/plugins/po.mdwn b/doc/plugins/po.mdwn
index cb0dd65c0..92fba349f 100644
--- a/doc/plugins/po.mdwn
+++ b/doc/plugins/po.mdwn
@@ -31,6 +31,12 @@ Example: `bla/page.fr.po` is the PO "message catalog" used to
 translate `bla/page.mdwn` into French; if `usedirs` is enabled, it is
 rendered as `bla/page/index.fr.html`, else as `bla/page.fr.html`
 
+(In)Compatibility
+=================
+
+This plugin does not support the `indexpages` mode. If you don't know
+what it is, you probably don't care.
+
 
 Configuration
 =============
@@ -219,6 +225,22 @@ interface could also be implemented at some point).
 If [[tips/untrusted_git_push]] is setup, one can edit the PO files in one's
 preferred `$EDITOR`, without needing to be online.
 
+Markup languages support
+------------------------
+
+Markdown is well supported. Some other markup languages supported by
+ikiwiki mostly work, but some pieces of syntax are not rendered
+correctly on the slave pages:
+
+* [[reStructuredText|rst]]: anonymous hyperlinks and internal
+  cross-references
+* [[wikitext]]: conversion of newlines to paragraphs
+* [[creole]]: verbatim text is wrapped, tables are broken
+* [[html]] and LaTeX: not supported yet; the dedicated po4a modules
+  could be used to support them, but they would need a security audit
+* other markup languages have not been tested.
+
+
 TODO
 ====
 
@@ -285,8 +307,6 @@ an initial goal, and analysing in detail the possible issues.
   read any file. NB: this hack depends on po4a internals to stay
   the same.
 
-#### To be checked
-
 ##### Locale::Po4a modules
 
 The modules we want to use have to be checked, as not all are safe
@@ -305,52 +325,19 @@ means the `Text` module only.
 
   > Freaky code, but seems ok due to use of `quotementa`.
 
+#### To be checked
+
 ##### Text::WrapI18N
 
 `Text::WrapI18N` can cause DoS (see the
 [Debian bug #470250](http://bugs.debian.org/470250)), but it is
 optional and we do not need the features it provides.
 
-It is loaded if available by `Locale::Po4a::Common`; looking at the
-code, I'm not sure we can prevent this at all, but maybe some symbol
-table manipulation tricks could work; overriding
-`Locale::Po4a::Common::wrapi18n` may be easier. I'm no expert at all
-in this field. Joey? [[--intrigeri]]
-
-> Update: Nicolas François suggests we add an option to po4a to
-> disable it. It would do the trick, but only for people running
-> a brand new po4a (probably too late for Lenny). Anyway, this option
-> would have to take effect in a `BEGIN` / `eval` that I'm not
-> familiar with. I can learn and do it, in case no Perl wizard
-> volunteers to provide the po4a patch. [[--intrigeri]]
-
->> That doesn't really need to be in a BEGIN. This patch moves it to
->> `import`, and makes this disable wrap18n:
->> `use Locale::Po4a::Common q{nowrapi18n}` --[[Joey]]
-
-<pre>
---- /usr/share/perl5/Locale/Po4a/Common.pm	2008-07-21 14:54:52.000000000 -0400
-+++ Common.pm	2008-11-11 18:27:34.000000000 -0500
-@@ -30,8 +30,16 @@
- use strict;
- use warnings;
- 
--BEGIN {
--    if (eval { require Text::WrapI18N }) {
-+sub import {
-+    my $class=shift;
-+    my $wrapi18n=1;
-+    if ($_[0] eq 'nowrapi18n') {
-+    	shift;
-+	$wrapi18n=0;
-+    }
-+    $class->export_to_level(1, $class, @_);
-+
-+    if ($wrapi18n && eval { require Text::WrapI18N }) {
-     
-         # Don't bother determining the wrap column if we cannot wrap.
-         my $col=$ENV{COLUMNS};
-</pre>
+> I proposed a patch based on Joey's to po4a-devel, allowing to fully
+> disable this module's use. When it is merged upstream, we'll need to add
+> `use Locale::Po4a::Common qw(nowrapi18n)` to `po.pm`, before loading
+> any other `Locale::Po4a` module. A versioned dependency may be needed.
+> --[[intrigeri]]
 
 ##### Term::ReadKey
 
@@ -359,42 +346,29 @@ works nicely without it. But the po4a Debian package recommends
 `libterm-readkey-perl`, so it will probably be installed on most
 systems using the po plugin.
 
-If `$ENV{COLUMNS}` is not set, `Locale::Po4a::Common` uses
-`Term::ReadKey::GetTerminalSize()` to get the terminal size. How safe
-is this?
+`Term::ReadKey` has too far reaching implications for us to
+be able to guarantee anything wrt. security.
 
-Part of `Term::ReadKey` is written in C. Depending on the runtime
-platform, this function use ioctl, environment, or C library function
-calls, and may end up running the `resize` command (without
-arguments).
+> The option that disables `Text::WrapI18N` also disables
+> `Term::ReadKey` as a consequence. [[--intrigeri]]
 
-IMHO, using Term::ReadKey has too far reaching implications for us to
-be able to guarantee anything wrt. security. Since it is anyway of no
-use in our case, I suggest we define `ENV{COLUMNS}` before loading
-`Locale::Po4a::Common`, just to be on the safe side. Joey?
-[[--intrigeri]]
+### msgmerge
 
-> Update: adding an option to disable `Text::WrapI18N`, as Nicolas
-> François suggested, would as a bonus disable `Term::ReadKey`
-> as well. [[--intrigeri]]
+`refreshpofiles()` runs this external program.
 
-### msgmerge
+A po4a developer answered he does "not expect any security issues from
+it". I did not manage to crash it with `zzuf`, nor was able to find
+any past security holes.
 
-`refreshpofiles()` runs this external program. A po4a developer
-answered he does "not expect any security issues from it".
+### msgfmt
 
-### Fuzzing input
+`isvalidpo()` runs this external program.
 
-I was not able to find any public information about gettext or po4a
-having been tested with a fuzzing program, such as `zzuf` or `fusil`.
-Moreover, some gettext parsers seem to be quite
-[easy to crash](http://fusil.hachoir.org/trac/browser/trunk/fuzzers/fusil-gettext),
-so it might be useful to bang msgmerge/po4a's heads against such
-a program in order to easily detect some of the most obvious DoS.
-[[--intrigeri]]
+* I could not manage to make it behave badly using zzuf, it exits
+  cleanly when too many errors are detected.
+* I could not find any past security holes.
 
-> po4a was not fuzzy-tested, but according to one of its developers,
-> "it would be really appreciated". [[--intrigeri]]
+### Fuzzing input
 
 Test conditions:
 
@@ -469,18 +443,9 @@ While:
 
 ... seems to lose the fight, at the `readpo(LICENSES.fr.po)` step,
 against some kind of infinite loop, deadlock, or any similar beast.
-It does not seem to eat memory, though.
-
-Whatever format module is used does not change anything. This is thus
-probably a bug in po4a's core or in a lib it depends on.
 
-The sub `read`, in `TransTractor.pm`, seems to be a good debugging
-starting point.
-
-#### msgmerge
-
-`msgmerge` is run in our `refreshpofiles` function. I did not manage
-to crash it with `zzuf`.
+The root of this bug lies in `Text::WrapI18N`, see above for
+possible solutions.
 
 gettext/po4a rough corners
 --------------------------
@@ -505,56 +470,47 @@ Better links
 
 ### Page title in links
 
-To use the page titles set with the [[meta|plugins/meta]] plugin when
-rendering links would be very much nicer, than the current
-"filename.LL" format. This is actually a duplicate for
-[[bugs/pagetitle_function_does_not_respect_meta_titles]].
-
-Going to work on this in my `meta` branch.
-
-### Translation status in links
-
-See [[contrib/po]].
-
-### Backlinks
-
-They are not updated when the source page changes (e.g. meta title).
-
-Page formats
-------------
+Using the fix to
+[[bugs/pagetitle_function_does_not_respect_meta_titles]] from
+[[intrigeri]]'s `meta` branch, the generated links' text is based on
+the page titles set with the [[meta|plugins/meta]] plugin. This has to
+be merged into ikiwiki upstream, though.
 
-Markdown is well supported, great, but what about others?
+Robustness tests
+----------------
 
-The [[po|plugins/po]] uses `Locale::Po4a::Text` for every page format;
-this can be expected to work out of the box with most other wiki-like
-formats supported by ikiwiki. Some of their ad-hoc syntax might be
-parsed in a strange way, but the worst problems I can imagine would be
-wrapping issues; e.g. there is code in po4a dedicated to prevent
-re-wrapping the underlined Markdown headers.
+### Enabling/disabling the plugin
 
-While it would be easy to better support formats such as [[html]] or
-LaTeX, by using for each one the dedicated po4a module, this can be
-problematic from a security point of view.
+- enabling the plugin with `po_translatable_pages` set to blacklist: **OK**
+- enabling the plugin with `po_translatable_pages` set to whitelist: **OK**
+- enabling the plugin without `po_translatable_pages` set: **OK**
+- disabling the plugin: **OK**
 
-**TODO**: test the more popular formats and write proper documentation
-about it.
+### Changing the plugin config
 
-Translation quality assurance
------------------------------
+- adding existing pages to `po_translatable_pages`: **OK**
+- removing existing pages from `po_translatable_pages`: **OK**
+- adding a language to `po_slave_languages`: **OK**
+- removing a language from `po_slave_languages`: **OK**
+- changing `po_master_language`: **OK**
+- replacing `po_master_language` with a language previously part of
+  `po_slave_languages`: needs two rebuilds, but **OK** (this is quite
+  a perverse test actually)
 
-Modifying a PO file via the CGI must be forbidden if the new version
-is not a valid PO file. As a bonus, check that it provides a more
-complete translation than the existing one.
+### Creating/deleting/renaming pages
 
-A new `cansave` type of hook would be needed to implement this.
+All cases of master/slave page creation/deletion/rename, both via RCS
+and via CGI, have been tested.
 
-Note: committing to the underlying repository is a way to bypass
-this check.
+### Misc
 
-Creating new pages on the web
------------------------------
+- general test with `usedirs` disabled: **OK**
+- general test with `indexpages` enabled: **not OK**
+- general test with `po_link_to=default` with `userdirs` enabled: **OK**
+- general test with `po_link_to=default` with `userdirs` disabled: **OK**
 
-See [[contrib/po]].
+Misc. bugs
+----------
 
 Documentation
 -------------