Bug: UTF-16 and UTF-32 are unhandled: New

[git.ikiwiki.info.git] / doc / users / smcv / gallery.mdwn

This plugin has now been implemented as [[plugins/contrib/album]]; this
page has older thoughts about it.

## Requirements

This plugin formats a collection of images into a photo gallery,
in the same way as many websites: good examples include the
PHP application [Gallery](http://gallery.menalto.com/), Flickr,
and Facebook's Photos "application".

The web UI I'm trying to achieve consists of one
[HTML page of thumbnails](http://www.pseudorandom.co.uk/2008/2008-03-08-panic-cell-gig/)
as an entry point to the gallery, where each thumbnail links to
[a "viewer" HTML page](http://www.pseudorandom.co.uk/2008/2008-03-08-panic-cell-gig/img_0068/)
with a full size image, next/previous thumbnail links, and
[[plugins/comments]].

(The Summer of Code [[plugins/contrib/gallery]] plugin does the
next/previous UI in Javascript using Lightbox, which means that
individual photos can't be bookmarked in a meaningful way, and
the best it can do as a fallback for non-Javascript browsers
is to provide a direct link to the image.)

Other features that would be good to have:

* minimizing the number of separate operations needed to make a gallery -
  editing one source file per gallery is acceptable, editing one
  source file per photo is not

* keeping photos outside source code control, for instance in an
  underlay

* assigning [[tags|ikiwiki/directive/tag]] to photos, providing a
  superset of Facebook's "show tagged photos of this person" functionality

* constructing galleries entirely via the web by uploading attachments

* inserting grouping (section headings) within a gallery; as in the example
  linked above, I'd like this to split up the thumbnails but not the
  next/previous trail

* rendering an `<object>/<embed>` arrangement to display videos, and possibly
  thumbnailing them in the same way as totem-video-thumbnailer
  (my camera can record short videos, so some of my web photo galleries
  contain them)

My plan is to have these directives:

* \[[!gallery]] registers the page it's on as a gallery, and displays all
  photos that are part of this gallery but not part of a \[[!gallerysection]]
  (below).

  All images (i.e. `*.png *.jpg *.gif`) that are attachments to the gallery page
  or its subpages are considered to be part of the gallery.

  Optional arguments:

  * filter="[[ikiwiki/PageSpec]]": only consider images to be part of the
    gallery if they also match this filter

  * sort="date|filename": order in which to sort the images

* \[[!gallerysection filter="[[ikiwiki/PageSpec]]"]] displays all photos in the
  gallery that match the filter

So,
[the gallery I'm using as an example](http://www.pseudorandom.co.uk/2008/2008-03-08-panic-cell-gig/)
could look something like this:

    \[[!gallery]]
    <!-- replaced with one uncategorized photo -->

    # Gamarra

    \[[!gallerysection filter="link(sometag)"]]
    <!-- all the Gamarra photos -->

    # Smokescreen

    \[[!gallerysection filter="link(someothertag)"]]
    <!-- all the Smokescreen photos -->

    <!-- ... -->

## Implementation ideas

The next/previous part this plugin overlaps with [[todo/wikitrails]].

A \[[!galleryimg]] directive to assign metadata to images might be necessary, so
the gallery page can contain something like:

    \[[!galleryimg p1010001.jpg title="..." caption="..." tags="foo"]]
    \[[!galleryimg p1010002.jpg title="..." caption="..." tags="foo bar"]]

However, allowing other pages to push in metadata like that will make
dependency tracking difficult.

Making the viewer pages could be rather tricky. Here are some options:
"synthesize source pages for viewers" is the one I'm leaning towards at the
moment.

### Viewers' source page is the gallery

One possibility is to write out the viewer pages as a side-effect of
preprocessing the \[[!gallery]] directive. The proof-of-concept implementation
below does this.  However, this does mean the viewer pages can't have tags or
metadata of their own and can't be matched by [[pagespecs|ikiwiki/pagespec]] or
[[wikilinks|ikiwiki/wikilink]].

It might be possible to implement tagging by using \[[!galleryimg]] to assign
the metadata to the *images* instead of their viewers; however, that would
require hacking up both `IkiWiki::htmllink` and `IkiWiki::urlto` to redirect
links to the image (e.g. from the \[[!map]] on a tag page) to become links to
the viewer page.

Modifications to the comments plugin would also be required, to make it allow
comments written to `foo/bar/comment_1._comment` even though the page foo/bar
does not really exist, and display comments on the viewer pages even though
they're not real pages. (Writing comments to `foo/bar.jpg/*._comment` is not
an option!)

### Synthesize source pages for viewers

(Edited to add: this is what [[plugins/contrib/album]] implements. --[[smcv]])

Another is to synthesize source pages for the viewers. This means they can have
tags and metadata, but trying to arrange for them to be scanned etc. correctly
without needing another refresh run is somewhat terrifying.
[[plugins/autoindex]] can safely create source pages because it runs in
the refresh hook, but I don't really like the idea of a refresh hook that scans
all source pages to see if they contain \[[!gallery]]...

The photo galleries I have at the moment, like the Panic Cell example above,
are made by using an external script to parse XML gallery descriptions (lists
of image filenames, with metadata such as titles), and using this to write
IkiWiki markup into a directory which is then used as an underlay. This is a
hack, but it works. The use of XML is left over from a previous attempt at
solving the same problem using Django.

Perhaps a better approach would be to have a setupfile option that names a
particular underlay directory (meeting the objective of not having large
photos under source code control) and generates a source page for each file
in that directory during the refresh hook. The source pages could be in the
underlay until they are edited (e.g. tagged), at which point they would be
copied into the source-code-controlled version in the usual way.

> Coming back to this: a specialized web UI to mark attachments as part of
> the gallery would make this easy too - you'd put the photos in the
> underlay, then go to the CGI and say "add all". --[[smcv]]

The synthetic source pages can be very simple, using the same trick as my
[[plugins/comments]] plugin (a dedicated [[directive|ikiwiki/directives]]
encapsulating everything the plugin needs). If the plugin automatically
gathers information like file size, pixel size, date etc. from the images, then
only the human-edited information and a filename reference need to be present
in the source page; with some clever lookup rules based on the filename of
the source page, not even the photo's filename is necessarily needed.

> Coming back to this later: the clever lookup rules make dependency tracking
> hard, though. --[[smcv]]

    \[[!meta title="..."]]
    \[[!meta date="..."]]
    \[[!meta copyright="..."]]
    \[[!tag ...]]

    \[[!galleryimageviewer p1010001.jpg]]

However, this would mean that editing tags and other metadata would require
editing pages individually. Rather than trying to "fix" that, perhaps it would
be better to have a special CGI interface for bulk tagging/metadata editing.
This could even be combined with a bulk upload form (a reasonable number of
file upload controls - maybe 20 - with metadata alongside each).

Uploading multiple images is necessarily awkward due to restrictions placed on
file upload controls by browsers for security reasons - sites like Facebook
allow whole directories to be uploaded at the same time, but they achieve this
by using a signed Java applet with privileged access to the user's filesystem.

I've found that it's often useful to be able to force the creation time of
photos (my camera's battery isn't very reliable, and it frequently decides that
the date is 0000-00-00 00:00:00), so treating the \[[!meta date]] of the source
page and the creation date of the photo as synonymous would be useful.

### Images are the viewer's source - special filename extension

Making the image be the source page (and generate HTML itself) would be
possible, but I wouldn't want to generate a HTML viewer for every `.jpg` on a
site, so either the images would have to have a special extension (awkward for
uploads from Windows users) or the plugin would have to be able to change
whether HTML was generated in some way (not currently possible).

### Images are the viewer's source - alter `ispage()`

It might be possible to hack up `ispage()` so some, but not all, images are
considered to "be a page":

* srcdir/not-a-photo.jpg → destdir/not-a-photo.jpg
* srcdir/gallery/photo.jpg → destdir/gallery/photo/index.html

Perhaps one way to do this would be for the photos to appear in a particular
underlay directory, which would also fulfil the objective of having photos not
be version-controlled:

* srcdir/not-a-photo.jpg → destdir/not-a-photo.jpg
* underlay/gallery/photo.jpg → destdir/gallery/photo/index.html

## Proof-of-concept implementation of "viewers' source page is the gallery"

    #!/usr/bin/perl
    package IkiWiki::Plugin::gallery;
    
    use warnings;
    use strict;
    use IkiWiki 2.00;
    
    sub import {
        hook(type => "getsetup", id => "gallery",  call => \&getsetup);
        hook(type => "checkconfig", id => "gallery", call => \&checkconfig);
        hook(type => "preprocess", id => "gallery",
                call => \&preprocess_gallery, scan => 1);
        hook(type => "preprocess", id => "gallerysection",
                call => \&preprocess_gallerysection, scan => 1);
        hook(type => "preprocess", id => "galleryimg",
                call => \&preprocess_galleryimg, scan => 1);
    }
    
    sub getsetup () {
        return
                plugin => {
                        safe => 1,
                        rebuild => undef,
                },
    }
    
    sub checkconfig () {
    }
    
    # page that is a gallery => array of images
    my %galleries;
    # page that is a gallery => array of filters
    my %sections;
    # page that is an image => page name of generated "viewer"
    my %viewers;
    
    sub preprocess_gallery {
        # \[[!gallery filter="!*/cover.jpg"]]
        my %params=@_;
    
        my $subpage = qr/^\Q$params{page}\E\//;
    
        my @images;
    
        foreach my $page (keys %pagesources) {
                # Reject anything not a subpage or attachment of this page
                next unless $page =~ $subpage;
    
                # Reject non-images
                # FIXME: hard-coded list of extensions
                next unless $page =~ /\.(jpg|gif|png|mov)$/;
    
                # Reject according to the filter, if any
                next if (exists $params{filter} &&
                        !pagespec_match($page, $params{filter},
                                location => $params{page}));
    
                # OK, we'll have that one
                push @images, $page;
    
                my $viewername = $page;
                $viewername =~ s/\.[^.]+$//;
                $viewers{$page} = $viewername;
    
                my $filename = htmlpage($viewername);
                will_render($params{page}, $filename);
        }
    
        $galleries{$params{page}} = \@images;
    
        # If we're just scanning, don't bother producing output
        return unless defined wantarray;
    
        # actually render the viewers
        foreach my $img (@images) {
                my $filename = htmlpage($viewers{$img});
                debug("rendering image viewer $filename for $img");
                writefile($filename, $config{destdir}, "# placeholder");
        }
    
        # display a list of "loose" images (those that are in no section);
        # this works because we collected the sections' filters during the
        # scan stage
    
        my @loose = @images;
    
        foreach my $filter (@{$sections{$params{page}}}) {
                my $_;
                @loose = grep { !pagespec_match($_, $filter,
                                location => $params{page}) } @loose;
        }
    
        my $_;
        my $ret = "<ul>\n";
        foreach my $img (@loose) {
                $ret .= "<li>";
                $ret .= "<a href=\"" . urlto($viewers{$img}, $params{page});
                $ret .= "\">$img</a></li>\n"
        }
        return "$ret</ul>\n";
    }
    
    sub preprocess_gallerysection {
        # \[[!gallerysection filter="friday/*"]]
        my %params=@_;
    
        # remember the filter for this section so the "loose images" section
        # won't include these images
        push @{$sections{$params{page}}}, $params{filter};
    
        # If we're just scanning, don't bother producing output
        return unless defined wantarray;
    
        # this relies on the fact that we ran preprocess_gallery once
        # already, during the scan stage
        my @images = @{$galleries{$params{page}}};
        @images = grep { pagespec_match($_, $params{filter},
                        location => $params{page}) } @images;
    
        my $_;
        my $ret = "<ul>\n";
        foreach my $img (@images) {
                $ret .= "<li>";
                $ret .= htmllink($params{page}, $params{destpage},
                        $viewers{$img});
                $ret .= "</li>";
        }
        return "$ret</ul>\n";
    }
    
    sub preprocess_galleryimg {
        # \[[!galleryimg p1010001.jpg title="" caption="" tags=""]]
        my $file = $_[0];
        my %params=@_;
    
        return "";
    }
    
    1