There's been a lot of work on contrib syntax highlighting plugins. One should be
picked and added to ikiwiki core.
-Ideally, it should support both converting whole source files into wiki
+> I'm calling this [[done]] since I added the [[plugins/highlight]]
+> plugin. There are some unresolved issues touched on here,
+> but they either have the own other bug reports, or are documented
+> as semi-features in the docs to the plugin. --[[Joey]]
+
+We want to support both converting whole source files into wiki
pages, as well as doing syntax highlighting as a preprocessor directive
-(which is either passed the text, or reads it from a file).
+(which is either passed the text, or reads it from a file). But,
+the [[ikiwiki/directive/format]] directive makes this easy enough to
+do if the plugin only supports whole source files. So, syntax plugins
+do no really need their own preprocessor directive, unless it makes
+things easier for the user.
## The big list of possibilities
-* [[plugins/contrib/highlightcode]] uses [[cpan Syntax::Highlight::Engine::Kate]],
+* [[plugins/contrib/highlightcode]] uses [[!cpan Syntax::Highlight::Engine::Kate]],
operates on whole source files only, has a few bugs (see
[here](http://u32.net/Highlight_Code_Plugin/), and needs to be updated to
- support [[bugs/multiple_pages_with_same_name]].
-* [[cpan IkiWiki-Plugin-syntax]] only operates as a directive.
+ support [[bugs/multiple_pages_with_same_name]]. (Currently a 404 :-( )
+* [[!cpan IkiWiki-Plugin-syntax]] only operates as a directive.
Interestingly, it supports multiple highlighting backends, including Kate
and Vim.
* [[plugins/contrib/syntax]] only operates as a directive
([[not_on_source_code_files|automatic_use_of_syntax_plugin_on_source_code_files]]),
- and uses [[cpan Text::VimColor]].
-* [[plugins/contrib/sourcehighlight]] uses src-highlight, and operates on
+ and uses [[!cpan Text::VimColor]].
+* [[plugins/contrib/sourcehighlight]] uses source-highlight, and operates on
whole source files only. Needs to be updated to
support [[bugs/multiple_pages_with_same_name]].
* [[sourcecode|todo/automatic_use_of_syntax_plugin_on_source_code_files/discussion]]
- also uses src-highlight, and operates on whole source files.
+ also uses source-highlight, and operates on whole source files.
Updated to work with the fix for [[bugs/multiple_pages_with_same_name]]. Untested with files with no extension, e.g. `Makefile`.
-* [[users/jasonblevins]]'s code plugin uses src-highlight, and supports both
- while file and directive use.
-
-## General problems
-
-* Using non-perl syntax highlighting backends is slow. I'd prefer either
- using a perl module, or a multiple-backend solution that can use a perl
- module as one option. (Or, if there's a great highlighter python module,
- we could use an external plugin..)
-* Currently no single plugin supports both modes of operation (directive
- and whole source file to page).
-
- > This is now fixed by the [[ikiwiki/directive/format]] directive for all
- > whole-source-file plugins, right?
-
+* [[users/jasonblevins]]'s code plugin uses source-highlight, and supports both
+ whole file and directive use.
+
+* [hlsimple](http://pivot.cs.unb.ca/git/?p=ikiplugins.git;a=blob_plain;f=IkiWiki/Plugin/hlsimple.pm;hb=HEAD) is a wrapper for the the perl module [[!cpan Syntax::Highlight::Engine::Simple]]. This is pure perl, pretty simple, uses css. It ought to be pretty fast (according to the author, and just because it is not external).
+On the other hand, there are not many predefined languages yet. Defining language syntaxes is about as much
+work as source-highlight, but in perl. I plan to package the base module for debian. Perhaps after the author
+releases the 5 or 6 language definitions he has running on his web site, it might be suitable for inclusion in ikiwiki. [[DavidBremner]]
+
+* [[plugins/highlight]] uses [highlight](http://www.andre-simon.de) via
+ its swig bindings. It optionally supports whole files, but also
+ integrates with the format directive to allow formatting of *any* of
+ highlight's supported formats. (For whole files, it uses either
+ keepextension or noextension, as appropriate for the type of file.)
+
+## General problems / requirements
+
+* Using non-perl syntax highlighting backends is slower. All things equal,
+ I'd prefer either using a perl module, or a multiple-backend solution that
+ can use a perl module as one option. (Or, if there's a great highlighter
+ python module, we could use an external plugin..)
+
+ Of course, some perl modules are also rather slow.. Kate, for example
+ can only process about 33 lines of C code, or 14 lines of
+ debian/changelog per second. That's **30 times slower than markdown**!
+
+ By comparison, source-highlight can do about 5000 lines of C code per
+ second... And launching the program 100 times on an empty file takes about
+ 5 seconds, which isn't bad. And, it has a C++ library, which it
+ seems likely perl bindings could be written for, to eliminate
+ even that overhead.
+ > [highlight](http://www.andre-simon.de) has similar features to source-highlight, and swig bindings
+ > that should make it trivial in principle to call from perl. I like highlight a bit better because
+ > it has a pass-through feature that I find very useful. My memory is unfortunately a bit fuzzy as to how
+ > well the swig bindings work. [[DavidBremner]]
+
+* Engines that already support a wide variety of file types are of
+ course preferred. If the engine doesn't support a particular type
+ of file, it could fall back to doing something simple like
+ adding line numbers. (IkiWiki-Plugin-syntax does this.)
+* XHTML output.
+* Emitting html that uses CSS to control the display is preferred,
+ since it allows for easy user customization. (Engine::Simple does
+ this; Kate can be configured to do it; source-highlight can be
+ made to do it via the switches `--css /dev/null --no-doc`)
* Nothing seems to support
[[wiki-formatted_comments|wiki-formatted_comments_with_syntax_plugin]]
inside source files. Doing this probably means post-processing the
results of the highlighting engine, to find places where it's highlighted
comments, and then running them through the ikiwiki rendering pipeline.
- This seems fairly doable with [[cpan Syntax::Highlight::Engine::Kate]],
+ This seems fairly doable with [[!cpan Syntax::Highlight::Engine::Kate]],
at least.
* The whole-file plugins tend to have a problem that things that look like
wikilinks in the source code get munged into links by ikiwiki, which can
* The whole-file plugins all get confused if there is a `foo.c` and a `foo.h`.
This is trivially fixable now by passing the keepextension option when
- registering the htmlize hooks, though.
+ registering the htmlize hooks, though. There's also a noextension option
+ that should handle the
+ case of source files with names that do not contain an extension (ie,
+ "Makefile") -- in this case you just register the while filename
+ in the htmlize hook.
* Whole-file plugins register a bunch of htmlize hooks. The wacky thing
about it is that, when creating a new page, you can then pick "c" or
- "h" or "pl" etc from the dropdown that normally has "mdwn" etc in it.
- Is this a bug, or a feature? (Even if a feature, plugins with many
- extensions make the dropdown unusable.. One way to deal with that is have
- a config setting that lists what extensions to offer highlighting for.
- Most people won't need/want the dozens some engines support.)
-* The per page highlighters can't handle creating wiki pages from
- "Makefile", or other files without a significant extension.
- Not clear how to fix this, as ikiwiki is very oriented toward file
- extensions. The workaround is to use a directive on a wiki page, pulling
- in the Makefile.
-
- > I wonder how hard it would be to make a patch whereby a file with
- > no `.` in the name, and a name that matches a filetype, and where
- > that filetype was registered `keepextension`, then the file is just
- > chosen as the appropriate type. This would allow `Makefile` to
- > work.
-
-like this:
-
- diff --git a/IkiWiki.pm b/IkiWiki.pm
- index 8d728c9..1bd46a9 100644
- --- a/IkiWiki.pm
- +++ b/IkiWiki.pm
- @@ -618,6 +618,8 @@ sub pagetype ($) { #{{{
-
- if ($page =~ /\.([^.]+)$/) {
- return $1 if exists $hooks{htmlize}{$1};
- + } elsif ($hooks{htmlize}{$page}{keepextension}) {
- + return $page;
- }
- return;
- } #}}}
-
-## format directive
-
-Rather than making syntax highlight plugins have to provide a preprocessor
-directive as well as handling whole source files, perhaps a generic format
-directive could be used:
-
- \[[!format pl """..."""]]
-
-That would run the text through the pl htmlizer, from the syntax hightligh
-plugin. OTOH, if "rst" were given, it would run the text through the rst
-htmlizer. So, more generic, allows mixing different types of markup on one
-page, as well as syntax highlighting. Does require specifying the type of
-format, instead of allowing it to be guessed (which some syntax highlighters
-can do). (This directive is now implemented..)
-
-Hmm, this would also allow comments inside source files to have mdwn
-embedded in them, without making the use of mdwn a special case, or needing
-to postprocess the syntax highlighter output to find comments.
-
- /* \[[!format mdwn """
-
- This is a comment in my C file. You can use mdwn in here.
-
- """]] */
+ "h" or "pl" etc from the dropdown that normally has "Markdown" etc in it.
+ Is this a bug, or a feature? Even if a feature, plugins with many
+ extensions make the dropdown unusable..
-Note that this assumes that directives are expanded in source files.
+ Perhaps the thing to do here is to use the new `longname` parameter to
+ the format hook, to give them all names that will group together at or
+ near the end of the list. Ie: "Syntax: perl", "Source code: c", etc.