Mail::Sendmail
CGI
Data::Dumper
-YAML
+YAML::XS
JSON
RPC::XML
my $class=shift;
my $content=shift;
- eval q{use YAML::Any};
- eval q{use YAML} if $@;
+ eval q{use YAML::XS};
die $@ if $@;
- $YAML::Syck::ImplicitUnicode=1;
IkiWiki::Setup::merge(Load(encode_utf8($content)));
}
my $type=shift;
my $prefix=shift;
- eval q{use YAML::Old};
- eval q{use YAML} if $@;
+ eval q{use YAML::XS};
die $@ if $@;
- $YAML::UseHeader=0;
+ $YAML::XS::QuoteNumericStrings=0;
- my $dump=Dump({$key => $value});
+ my $dump=decode_utf8(Dump({$key => $value}));
+ $dump=~s/^---\n//; # yaml header, we don't want
chomp $dump;
if (length $prefix) {
$dump=join("\n", map { $prefix.$_ } split(/\n/, $dump));
* mdwn: Added nodiscount setting, which can be used to avoid using the
markdown discount engine, when maximum compatability is needed.
+ * Switch to YAML::XS to work around insanity in YAML::Mo. Closes: #657533
-- Joey Hess <joeyh@debian.org> Mon, 16 Jan 2012 13:41:14 -0400
libtimedate-perl, libhtml-template-perl,
libhtml-scrubber-perl, wdg-html-validator,
libhtml-parser-perl, liburi-perl (>= 1.36), perlmagick, po4a (>= 0.34),
- libfile-chdir-perl, libyaml-perl, python-support
+ libfile-chdir-perl, libyaml-libyaml-perl, python-support
Maintainer: Joey Hess <joeyh@debian.org>
Uploaders: Josh Triplett <josh@freedesktop.org>
Standards-Version: 3.9.2
Depends: ${misc:Depends}, ${perl:Depends}, ${python:Depends},
libtext-markdown-discount-perl,
libhtml-scrubber-perl, libhtml-template-perl,
- libhtml-parser-perl, liburi-perl (>= 1.36), libyaml-perl, libjson-perl
+ libhtml-parser-perl, liburi-perl (>= 1.36), libyaml-libyaml-perl, libjson-perl
Recommends: gcc | c-compiler,
libc6-dev | libc-dev,
git (>= 1:1.7) | git-core (>= 1:1.5.0) | subversion | tla | bzr (>= 0.91) | mercurial | monotone (>= 0.38) | darcs,
--- /dev/null
+Hello,
+
+I studied this [[guy's problem|forum/Encoding_problem_in_french_with_ikiwiki-calendar]] and I propose here a (dirty) hack to correct it.
+
+Bug summary: when using the [[calendar plugin|plugins/calendar]] in French (`LANG=fr_FR.UTF-8`), "Décembre" (French for "December") is rendered as "Décembre".
+
+I managed to track this problem down to an encoding problem of `POSIX::strftime` in `Ikiwiki/Plugin/calendar.pm`. I used [[this guy's solution|http://www.perlmonks.org/?node_id=857018]] to solve the problem (the diff is printed below).
+
+The problem is that I do not know Perl, encoding is one of the thing I would be happy not to dive into, and it is the first time I contribute to Ikiwiki: I copied and made a few changes to the code I found without understanding it. So I am not sure that my code is neat, or works in every situation. Feel free to (help me to) improve it!
+
+Cheers,
+Louis
+
+
+ diff --git a/IkiWiki/Plugin/calendar.pm b/IkiWiki/Plugin/calendar.pm
+ index c7d2b7c..1345939 100644
+ --- a/IkiWiki/Plugin/calendar.pm
+ +++ b/IkiWiki/Plugin/calendar.pm
+ @@ -22,7 +22,14 @@ use warnings;
+ use strict;
+ use IkiWiki 3.00;
+ use Time::Local;
+ -use POSIX ();
+ +
+ +use POSIX qw/setlocale LC_TIME strftime/;
+ +use Encode;
+ +my ($strftime_encoding)= setlocale(LC_TIME)=~m#\.([^@]+)#;
+ +sub strftime_utf8 {
+ +# try to return an utf8 value from strftime
+ + $strftime_encoding ? Encode::decode($strftime_encoding, &strftime) : &strftime;
+ +}
+
+ my $time=time;
+ my @now=localtime($time);
+ @@ -123,10 +130,10 @@ sub format_month (@) {
+ }
+
+ # Find out month names for this, next, and previous months
+ - my $monthabbrev=POSIX::strftime("%b", @monthstart);
+ - my $monthname=POSIX::strftime("%B", @monthstart);
+ - my $pmonthname=POSIX::strftime("%B", localtime(timelocal(0,0,0,1,$pmonth-1,$pyear-1900)));
+ - my $nmonthname=POSIX::strftime("%B", localtime(timelocal(0,0,0,1,$nmonth-1,$nyear-1900)));
+ + my $monthabbrev=strftime_utf8("%b", @monthstart);
+ + my $monthname=strftime_utf8("%B", @monthstart);
+ + my $pmonthname=strftime_utf8("%B", localtime(timelocal(0,0,0,1,$pmonth-1,$pyear-1900)));
+ + my $nmonthname=strftime_utf8("%B", localtime(timelocal(0,0,0,1,$nmonth-1,$nyear-1900)));
+
+ my $archivebase = 'archives';
+ $archivebase = $config{archivebase} if defined $config{archivebase};
+ @@ -182,7 +189,7 @@ EOF
+ my %dowabbr;
+ for my $dow ($week_start_day..$week_start_day+6) {
+ my @day=localtime(timelocal(0,0,0,$start_day++,$params{month}-1,$params{year}-1900));
+ - my $downame = POSIX::strftime("%A", @day);
+ + my $downame = strftime_utf8("%A", @day);
+ my $dowabbr = substr($downame, 0, 1);
+ $downame{$dow % 7}=$downame;
+ $dowabbr{$dow % 7}=$dowabbr;
+ @@ -329,8 +336,8 @@ EOF
+ for (my $month = 1; $month <= 12; $month++) {
+ my @day=localtime(timelocal(0,0,0,15,$month-1,$params{year}-1900));
+ my $murl;
+ - my $monthname = POSIX::strftime("%B", @day);
+ - my $monthabbr = POSIX::strftime("%b", @day);
+ + my $monthname = strftime_utf8("%B", @day);
+ + my $monthabbr = strftime_utf8("%b", @day);
+ $calendar.=qq{\t<tr>\n} if ($month % $params{months_per_row} == 1);
+ my $tag;
+ my $mtag=sprintf("%02d", $month);
Note that for more formal bug reports or todo items, you can also edit the
[[bugs]] and [[todo]] pages.
+
## Current topics ##
[[!inline pages="forum/* and !forum/discussion and !forum/*/*"
--- /dev/null
+Hi!
+
+I'm using the ikiwiki calendar plugin.
+
+My website is in french (locale fr_FR.UTF-8), and calendars that are generated by the plugin makes some encodi$
+
+I don't know how the plugin generate translation for dates, but I've seen that there is no ikiwiki translation$
+
+That's why I suppose (but I'm not sure) that it use date unix command to insert date into the html page, witho$
+
+Could I have forgotten some options to make it nice or not?
+
+Is someone could test it and verify if it works or not?
+
+Thanks.
+
+Zut
--- /dev/null
+Just thought people might like to know I've added a couple more plugins to contrib.
+
+[[plugins/contrib/newpage]]: This plugin adds a new action to the "ACTIONS" section of a page; a button labelled "create" and an input field next to it.
+
+The common way of creating a new page is to edit a different page and add a link to the new page. However, there are some situations where that is a nuisance; for example, where pages are listed using a map directive. The newpage plugin enables one to simply type the name of the new page, click the "create" button, and one is then taken to the standard IkiWiki create-page form.
+
+[[plugins/contrib/jssearchfield]]: This plugin provides the [[plugins/contrib/ikiwiki/directive/jssearchfield]] directive. This
+enables one to search the structured data ("field" values) of multiple pages.
+This uses Javascript for the searching, which means that the entire thing
+is self-contained and does not require a server or CGI access, unlike
+the default IkiWiki search. This means that it can be used in places such
+as ebook readers. The disadvantage is that because Javascript runs
+in the browser, the searching is only as fast as the machine your browser
+is running on.
+
+Because this uses Javascript, the htmlscrubber must be turned off for any page where the directive is used.
+
+This plugin depends on the [[!iki plugins/contrib/field]] plugin.
+
+--[[KathrynAndersen]]
* [Richard "RichiH" Hartmann](http://richardhartmann.de/blog) - I thought I had added myself a year ago. Oups :)
* [Jonas Smedegaard](http://dr.jones.dk/) multilingual "classic" website w/ blog
* [Siri Reiter](http://sirireiter.dk/) portfolio website with a blog (in danish)
-* [L'Altro Wiki](http://laltromondo.dynalias.net/~iki/) Tutorials, reviews, miscellaneus articles in English and Italian, from the IRC network syrolnet.org
+* [L'Altro Wiki](http://laltromondo.dynalias.net/~iki/) Tutorials, reviews, miscellaneus articles in English and Italian.
* [STUPiD](http://lhzhang.com/)
* gregoa's [p.r. - political rants](http://info.comodo.priv.at/pr/)
* [Michael Hammer](http://www.michael-hammer.at/)
--- /dev/null
+The `jssearchfield` directive is supplied by the [[!iki plugins/contrib/jssearchfield desc=jssearchfield]] plugin.
+
+This enables one to search the structured data ("field" values) of
+multiple pages. A search form is constructed, and the searching is
+done with Javascript, which means that the entire thing is self-contained.
+This depends on the [[!iki plugins/contrib/field]] plugin.
+
+The pages to search are selected by a PageSpec given by the "pages"
+parameter.
+The fields to search are given by the "fields" parameter. By default,
+the field name is given, and the user can type the search parameter for
+that field into a text input field.
+
+## OPTIONS
+
+**pages**: A PageSpec to determine the pages to search through.
+
+**fields**: The fields to put into the search form, and to display
+in the results.
+
+**tagfields**: Display the given fields as a list of tags that can
+be selected from, rather than having a text input field. Every distinct
+value of that field will be listed, so it is best used for things with
+short values, like "Author" rather than long ones like "Description".
+Note that "tagfields" must be a subset of "fields".
+
+**sort**: A SortSpec to determine how the matching pages should be sorted; this is the "default" sort order that the results will be displayed in.
+The search form also gives the option of "random" sort, which will
+display the search results in random order.
+
+## SEARCHING
+
+The search form that is created by this directive contains the following:
+
+* for each search field, a label, plus either a text input field, or a list of checkboxes with values next to them if the field is also a tagfield. Note that the lists of checkboxes are initially hidden; one must click on the triangle next to the label to display them.
+* a "sort" toggle. One can select either "default" or "random".
+* A "Search!" button, to trigger the search if needed (see below)
+* A "Reset" button, which will clear all the values.
+
+The searching is dynamic. As soon as a value is changed, either by tabbing out of the text field, or by selecting or de-selecting a checkbox, the search
+results are updated. Furthermore, for tagfields, the tagfield lists
+themselves are updated to reflect the current search results.
--- /dev/null
+[[!template id=plugin name=jssearchfield author="[[rubykat]]"]]
+[[!tag type/search]]
+IkiWiki::Plugin::jssearchfield - Create a search form to search page field data.
+
+This plugin provides the [[ikiwiki/directive/jssearchfield]] directive. This
+enables one to search the structured data ("field" values) of multiple pages.
+This uses Javascript for the searching, which means that the entire thing
+is self-contained and does not require a server or CGI access, unlike
+the default IkiWiki search. This means that it can be used in places such
+as ebook readers. The disadvantage is that because Javascript runs
+in the browser, the searching is only as fast as the machine your browser
+is running on.
+
+Because this uses Javascript, the htmlscrubber must be turned off for any page where the directive is used.
+
+This plugin depends on the [[!iki plugins/contrib/field]] plugin.
+
+## Activate the plugin
+
+ # activate the plugin
+ add_plugins => [qw{goodstuff field jssearchfield ....}],
+
+ # disable scrubbing for search page
+ htmlscrubber_skip => 'mysearchpage',
+
+## PREREQUISITES
+
+ IkiWiki
+ IkiWiki::Plugin::field
+ HTML::Template
+
+## DOWNLOAD
+
+* browse at GitHub: <http://github.com/rubykat/ikiplugins/blob/master/IkiWiki/Plugin/jssearchfield.pm>
+* git repo at git://github.com/rubykat/ikiplugins.git
> It does not, however, have a markdown to html converter -- for
> previewing it has to talk to the server with AJAX.
> --[[Joey]]
+
+>> I've got pagedown working on my personal site (simon.kisikew.org) but I'm not sure how
+>> I can inject the relevant <div>'s in the right place. They need to go **above**
+>> the editing <textarea> . (Too bad about the licensing, it's rather nice.)
+>> I had to do one minor change to it to have it inject itself into the page properly,
+>> and that was to make this change in `Markdown.Editor.js`:
+>>
+>> `this.input = doc.getElementById("editcontent" + postfix);`
+>>
+>> on line 247. --[[simonraven]]
+
+>>> Well, I re-figured out that I needed a TMPL_VAR FOO in the template(s). --[[simonraven]]
Before I go writing a whole swack of test cases, could you merge
my latest? Through at least ad0e56cdcaaf76bc68d1b5c56e6845307b51c44a
there should be no functional change. --[[schmonz]]
+
+Never mind, I was able to convince myself (by cloning `origin`
+afresh and merging from `schmonz/cvs`). The history is a little
+gross but the before-and-after diff looks right.
+
+Bugs found and fixed so far:
+
+* Stop treating text files as binary (`-kb`) on `rcs_add()`
+ (ac8eab29e8394aca4c0b23a6687ec947ea1ac869)
* [[!shortcut name=archive url="http://web.archive.org/*/%S"]]
* [[!shortcut name=gmap url="https://maps.google.com/maps?q=%s"]]
* [[!shortcut name=gmsg url="https://groups.google.com/groups?selm=%s"]]
-* [[!shortcut name=wikipedia url="https://en.wikimedia.org/wiki/%s"]]
+* [[!shortcut name=wikipedia url="https://en.wikipedia.org/wiki/%s"]]
* [[!shortcut name=wikitravel url="https://wikitravel.org/en/%s"]]
* [[!shortcut name=wiktionary url="https://en.wiktionary.org/wiki/%s"]]
* [[!shortcut name=debbug url="http://bugs.debian.org/%S" desc="Debian bug #%s"]]
We could use a big machine, with plenty of CPUs. Could some multi-threading support be added to ikiwiki, by forking out all the external heavy plugins (imagemagick, tex, ...) and/or by processing pages in parallel?
Disclaimer: I know nothing of the Perl approach to parallel processing.
+
+> I agree that it would be lovely to be able to use multiple processors to speed up rebuilds on big sites (I have a big site myself), but, taking a quick look at what Perl threads entails, and taking into acount what I've seen of the code of IkiWiki, it would take a massive rewrite to make IkiWiki thread-safe - the API would have to be completely rewritten - and then more work again to introduce threading itself. So my unofficial humble opinion is that it's unlikely to be done.
+> Which is a pity, and I hope I'm mistaken about it.
+> --[[KathrynAndersen]]
+
+> > I have much less experience with the internals of Ikiwiki, much
+> > less Multi-threading perl, but I agree that to make Ikiwiki thread
+> > safe and to make the modifications to really take advantage of the
+> > threads is probably beyond the realm of reasonable
+> > expectations. Having said that, I wonder if there aren't ways to
+> > make Ikiwiki perform better for these big cases where the only
+> > option is to wait for it to grind through everything. Something
+> > along the lines of doing all of the aggregation and dependency
+> > heavy stuff early on, and then doing all of the page rendering
+> > stuff at the end quasi-asynchronously? Or am I way off in the deep
+> > end.
+> >
+> > From a practical perspective, it seems like these massive rebuild
+> > situations represent a really small subset of ikiwiki builds. Most
+> > sites are pretty small, and most sites need full rebuilds very
+> > very infrequently. In that scope, 10 minute rebuilds aren't that
+> > bad seeming. In terms of performance challenges, it's the one page
+> > with 3-5 dependency that takes 10 seconds (say) to rebuild that's
+> > a larger challenge for Ikiwiki as a whole. At the same time, I'd
+> > be willing to bet that performance benefits for these really big
+> > repositories for using fast disks (i.e. SSDs) could probably just
+> > about meet the benefit of most of the threading/async work.
+> >
+> > --[[tychoish]]
+
+>>> It's at this point that doing profiling for a particular site would come
+>>> in, because it would depend on the site content and how exactly IkiWiki is
+>>> being used as to what the performance bottlenecks would be. For the
+>>> original poster, it would be image processing. For me, it tends to be
+>>> PageSpecs, because I have a lot of maps and reports.
+
+>>> But I sincerely don't think that Disk I/O is the main bottleneck, not when
+>>> the original poster mentions CPU usage, and also in my experience, I see
+>>> IkiWiki chewing up 100% CPU usage one CPU, while the others remain idle. I
+>>> haven't noticed slowdowns due to waiting for disk I/O, whether that be a
+>>> system with HD or SSD storage.
+
+>>> I agree that large sites are probably not the most common use-case, but it
+>>> can be a chicken-and-egg situation with large sites and complete rebuilds,
+>>> since it can often be the case with a large site that rebuilding based on
+>>> dependencies takes *longer* than rebuilding the site from scratch, simply
+>>> because there are so many pages that are interdependent. It's not always
+>>> the number of pages itself, but how the site is being used. If IkiWiki is
+>>> used with the absolute minimum number of page-dependencies - that is, no
+>>> maps, no sitemaps, no trails, no tags, no backlinks, no albums - then one
+>>> can have a very large number of pages without having performance problems.
+>>> But when you have a change in PageA affecting PageB which affects PageC,
+>>> PageD, PageE and PageF, then performance can drop off horribly. And it's a
+>>> trade-off, because having features that interlink pages automatically is
+>>> really nifty ad useful - but they have a price.
+
+>>> I'm not really sure what the best solution is. Me, I profile my IkiWiki builds and try to tweak performance for them... but there's only so much I can do.
+>>> --[[KathrynAndersen]]
+
+>>>> IMHO, the best way to get a multithreaded ikiwiki is to rewrite it
+>>>> in haskell, using as much pure code as possible. Many avenues
+>>>> then would open up to taking advantage of haskell's ability to
+>>>> parallize pure code.
+>>>>
+>>>> With that said, we already have some nice invariants that could be
+>>>> used to parallelize page builds. In particular, we know that
+>>>> page A never needs state built up while building page B, for any
+>>>> pages A and B that don't have a dependency relationship -- and ikiwiki
+>>>> tracks such dependency relationships, although not currently in a form
+>>>> that makes it very easy (or fast..) to pick out such groups of
+>>>> unrelated pages.
+>>>>
+>>>> OTOH, there are problems.. building page A can result in changes to
+>>>> ikiwiki's state; building page B can result in other changes. All
+>>>> such changes would have to be made thread-safely. And would the
+>>>> resulting lock contention result in a program that ran any faster
+>>>> once parallelized?
+>>>>
+>>>> Which is why [[rewrite_ikiwiki_in_haskell]], while pretty insane, is
+>>>> something I keep thinking about. If only I had a spare year..
+>>>> --[[Joey]]
a bunch of haskell libraries. OTOH, it might be possible to build a
static binary at home and upload it, thus avoiding a messy installation
procedure entirely.
-* I can barely code in haskell yet. I'm probably about 100x faster at
- programming in perl. I need to get some more practical experience before
- I´m fast and seasoned enough in haskell to attempt such a project.
- (And so far, progress at learning has been slow and I have not managed
- to write anything serious in haskell.) --[[Joey]]
+ --[[Joey]]
>>> It doesn't really. I recently (re-)read about couchdb and thought that
>>> what it was trying to do had some comparisons with the thinking going on
>>> in [[todo/structured_page_data]]. -- [[Jon]]
+
+-----
+
+I'm torn about this idea, if it's actually serious. I'm very comfortable
+programming in Perl, and have written quite a few modules for IkiWiki, and
+it would be a huge pain to have to start from scratch all over again. On
+the other hand, this could be a motivation for me to learn Haskell. My
+only encounter with Haskell has been a brief time when I was using the
+Xmonad window manager, but it looks like an interesting language.
+Functional programming is cool.
+
+There are a lot of interesting plusses for Haskell you note (in the parent
+page), but it's true that the idea is horribly daunting (as [[Joey]] said
+"If only I had a spare year"). Is there any way that you could "start
+small"? Because nothing will ever happen if the task is too daunting to
+even start.
+
+> This seems destined to remain a thought experiment unless something like
+> that can be done, or I get a serious case of second system disease.
+>
+> I've considered doing things like using the external plugin interface
+> to run a separate haskell program, which would allow implementing
+> arbitrary plugins in haskell (starting with a pandoc plugin..),
+> and could perhaps grow to subsume the perl code. However, this would
+> stick us with the perl data structures, which are not a very good fit
+> for haskell. --[[Joey]]
+
+On further thought... perhaps it would be easier to fork or contribute to
+an existing Haskell-based wiki, such as <a
+href="http://jaspervdj.be/hakyll">Hakyll</a>?
+
+--[[KathrynAndersen]]
+
+> As far as I know there are no other wikis (haskell or otherwise)
+> that are wiki compilers. Since we know from experience that dealing
+> with static compilation turns out to be one of the trickiest parts of
+> ikiwiki, I'm doubtful about trying to bolt that into one. --[[Joey]]
if (! defined $var || ! defined $val) {
die gettext("usage: --set-yaml var=value"), "\n";
}
- eval q{use YAML::Any};
- eval q{use YAML} if $@;
+ eval q{use YAML::XS; use Encode};
die $@ if $@;
- eval q{$YAML::Syck::ImplicitUnicode=1};
- $config{$var}=Load($val."\n");
+ $config{$var}=Load(encode_utf8($val)."\n");
},
"version" => sub {
print "ikiwiki version $IkiWiki::version\n";