cherry-pick uri security fix

[git.ikiwiki.info.git] / doc / index / discussion.mdwn

Seems like there should be a page for you to post your thoughts about
ikiwiki, both pro and con, anything that didn't work, ideas, or whatever.
Do so here..

Note that for more formal bug reports or todo items, you can also edit the
[[bugs]] and [[todo]] pages.

[[toc ]]

# Installation/Setup questions

Ikiwiki creates a .ikiwiki directory in my wikiwc working directory. Should I
"svn add .ikiwiki" or add it to svn:ignore?

> `.ikiwiki` is used by ikiwiki to store internal state. You can add it to
> svn:ignore. --[[Joey]]
> > Thanks a lot.

Is there an easy way to log via e-mail to some webmaster address, instead
of via syslog?

> Not sure why you'd want to do that, but couldn't you use a tool like
> logwatch to mail selected lines from the syslog? --[[Joey]]

> > The reason is that I'm not logged in on the web server regularly to
> > check the log files. I'll see whether I can install a logwatch instance.

----

# Installation of selected docs (html)

The latest release has around 560 files (over 2MB) in html.

Any suggestions or ideas on limiting what html is installed?

For example, I don't see value in every ikiwiki install out there to also install personal "users" ikiwiki pages.

For now I copy ikiwiki.setup. And then use pax with -L switch to copy the targets of the symlinks of the basewiki.

I was thinking of making a list of desired documents from the html directory to install.

--JeremyReed

> You don't need any of them, unless you want to read ikiwiki's docs locally.
> 
> I don't understand why you're installing the basewiki files manually;
> ikiwiki has a Makefile that will do this for you. --[[Joey]]

>> The Makefile's install doesn't do what I want so I use different installer for it.
>> It assumes wrong location for man pages for me. (And it should consider using INSTALLVENDORMAN1DIR and
>> MAN1EXT but I don't know about section 8 since I don't know of perl value for that.)
>> I don't want w3m cgi installed; it is optional for my package.
>> I will just patch for that instead of using my own installer.
>> Note: I am working on the pkgsrc package build specification for this. This is for creating
>> packages for NetBSD, DragonFly and other systems that use pkgsrc package system.
>> --JeremyReed

# Installation as non-root user

I'd like to install ikiwiki as a non-root user. I can plow through getting all the 
perl dependencies installed because that's well documented in the perl world,
but I don't know how to tell ikiwiki to install somewhere other than / --BrianWilson

----

# Upgrade steps

I upgrades from 1.40  to 2.6.1. I ran "ikiwiki --setup" using my existing ikiwiki.setup configuration.
I had many errors like:

    /home/bsdwiki/www/wiki/wikilink/index.html independently created, not overwriting with version from wikilink
    BEGIN failed--compilation aborted at (eval 5) line 129.

and:

    failed renaming /home/bsdwiki/www/wiki/smileys.ikiwiki-new to /home/bsdwiki/www/wiki/smileys: Is a directory
    BEGIN failed--compilation aborted at (eval 5) line 129.

Probably about six errors like this. I worked around this by removing the files and directories it complained about.
Finally it finished.

> As of version 2.0, ikiwiki enables usedirs by default. See
> [[tips/switching_to_usedirs]] for details. --[[Joey]]

>> I read the config wrong. I was thinking that it showed the defaults even though commented out
>> (like ssh configs do). I fixed that part. --JeremyReed

My next problem was that ikiwiki start letting me edit without any password authentication. It used to prompt
me for a password but now just goes right into the "editing" mode.
The release notes for 2.0 say password auth is still on by default.

> It sounds like you have the anonok plugin enabled?

>> Where is the default documented? My config doesn't have it uncommented.

The third problem is that when editing my textbox is empty -- no content.

This is using my custom rcs.pm which has been used thousands of times.

> Have you rebuilt the cgi wrapper since you upgraded ikiwiki? AFAIK I
> fixed a bug that could result in the edit box always being empty back in
> version 2.3. The only other way it could happen is if ikiwiki does not
> have saved state about the page that it's editing (in .ikiwiki/index).

>> Rebuilt it several times. Now that I think of it, I think my early problem of having
>> no content in the textbox was before I rebuilt the cgi. And after I rebuilt the whole webpage was empty.

Now I regenerated my ikiwiki.cgi again (no change to my configuration,
and I just get an empty HTML page when attempting editing or "create".

> If the page is completly empty then ikiwiki is crashing before it can
> output anything, though this seems unlikely. Check the webserver logs.

Now I see it created directories for my data. I fixed that by setting 
usedirs (I see that is in the release notes for 2.0) and rerunning ikiwiki --setup
but I still have empty pages for editing (no textbox no html at all).

> Is IkiWiki crashing? If so, it would probably leave error text in the apache logs. --[[TaylorKillian]]

>> Not using apache. Nothing useful in logs other thn the HTTP return codes are "0" and bytes is "-"
>> on the empty ikiwiki.cgi output (should say " 200 " followed by bytes).

>>> You need to either figure out what your web server does with stderr
>>> from cgi programs, or run ikiwiki.cgi at the command line with an
>>> appropriate environment so it thinks it's being called from a web
>>> server, so you can see how it's failing. --[[Joey]]

(I am posting this now, but will do some research and post some more.)

Is there any webpage with upgrade steps?

> Users are expected to read [[news]], which points out any incompatible
> changes or cases where manual action is needed.

>> I read it but read the usedirs option wrong :(.
>> Also it appears to be missing the news from between 1.40 to 2.0 unless they dont' exist.
>> If they do exist maybe they have release notes I need?

>>> All the old ones are in the NEWS file. --[[Joey]]

--JeremyReed

My followup: I used a new ikiwiki.setup based on the latest version. But no changes for me.

Also I forgot to mention that do=recentchanges works good for me. It uses my 
rcs_recentchanges in my rcs perl module.

The do=prefs does nothing though -- just a blank webpage.

> You need to figure out why ikiwiki is crashing. The webserver logs should
> tell you.

I also set verbose => 1 and running ikiwiki --setup was verbose, but no changes in running CGI.
I was hoping for some output.

I am guessing that my rcs perl module stopped working on the upgrade. I didn't notice any release notes
on changes to revision control modules. Has something changed? I will also look.

> No, the rcs interface has not needed to change in a long time. Also,
> nothing is done with the rcs for do=prefs.

>> Thanks. I also checked differences between 1.40 Rcs plugins and didn't notice anything significant.

--JeremyReed

Another Followup: I created a new ikiwiki configuration and did the --setup to 
create an entirely different website. I have same problem there. No prompt for password
and empty webpage when using the cgi.
I never upgraded any perl modules so maybe a new perl module is required but I don't see any errors so I don't know.

The only errors I see when building and installing ikiwiki are:

    Can't exec "otl2html": No such file or directory at IkiWiki/Plugin/otl.pm line 66.

    gettext 0.14 too old, not updating the pot file

I don't use GNU gettext on here.

I may need to revert back to my old ikiwiki install which has been used to thousands of times (with around
1000 rcs commits via ikiwiki).

--JeremyReed

I downgraded to version 1.40 (that was what I had before I wrote wrong above).
Now ikiwiki is working for me again (but using 1.40). I shouldn't have tested on production system :)

--JeremyReed

I am back. On a different system, I installed ikiwiki 2.6.1. Same problem -- blank CGI webpage.

So I manually ran with:

    REQUEST_METHOD=GET QUERY_STRING='do=create&page=jcr' kiwiki.cgi

And clearly saw the error:

    [IkiWiki::main] Fatal: Bad template engine CGI::FormBuilder::Template::div: Can't locate CGI/FormBuilder/Template/div.pm

So I found my version was too old and 3.05 is the first to provide "Div" support. I upgraded my p5-CGI-FormBuilder to 3.0501.
And ikiwiki CGI started working for me.

The Ikiwiki docs about this requirement got removed in Revision 4367. There should be a page that lists the requirements.
(I guess I could have used the debian/control file.)

> There is a page, [[install]] documents that 3.05 is needed.

>> Sorry, I missed that. With hundreds of wikipages it is hard to read all of them.
>> I am updating the download page now to link to it.

I am now using ikiwiki 2.6.1 on my testing system.

--JeremyReed

----
# Excellent - how do I translate a TWiki site?

I just discovered ikiwiki quite by chance, I was looking for a console/terminal
menu system and found pdmenu.  So pdmenu brought me to here and I've found ikiwiki!
It looks as if it's just what I've been wanting for a long time.  I wanted something
to create mostly text web pages which, as far as possible, have source which is human 
readable or at least in a standard format.  ikiwiki does this twice over by using 
markdown for the source and producing static HTML from it.

I'm currently using TWiki and have a fair number of pages in that format, does
anyone have any bright ideas for translating?  I can knock up awk scripts fairly
easily, perl is possible (but I'm not strong in perl).

> Let us know if you come up with something to transition from the other
> format. Another option would be writing a ikiwiki plugin to support the
> TWiki format. --[[Joey]]

> Jamey Sharp and I have a set of scripts in progress to convert other wikis to ikiwiki, including history, so that we can migrate a few of our wikis.  We already have support for migrating MoinMoin wikis to ikiwiki, including conversion of the entire history to Git.  We used this to convert the [XCB wiki](http://xcb.freedesktop.org/wiki/) to ikiwiki; until we finalize the conversion and put the new wiki in place of the old one, you can browse the converted result at <http://xcb.freedesktop.org/ikiwiki>.  We already plan to add support for TWiki (including history, since you can just run parsecvs on the TWiki RCS files to get Git), so that we can convert the [Portland State Aerospace Society wiki](http://psas.pdx.edu) (currently in Moin, but with much of its history in TWiki, and with many of its pages still in TWiki format using Jamey's TWiki format for MoinMoin).
>
> Our scripts convert by way of HTML, using portions of the source wiki's code to render as HTML (with some additional code to do things like translate MoinMoin's `\[[TableOfContents]]` to ikiwiki's `\[[toc ]]`), and then using a modified [[cpan HTML::WikiConverter]] to turn this into markdown and ikiwiki.  This produces quite satisfactory results, apart from things that don't have any markdown equivalent and thus remain HTML, such as tables and definition lists.  Conversion of the history occurs by first using another script we wrote to translate MoinMoin history to Git, then using our git-map script to map a transformation over the Git history.
>
> We will post the scripts as soon as we have them complete enough to convert our wikis.
>
> -- [[JoshTriplett]]

>> Thanks for an excellent Xmas present, I will appreciate the additional 
>> users this will help switch to ikiwiki! --[[Joey]]


>> Sounds great indeed. Learning from [here](http://www.bddebian.com/~wiki/AboutTheTWikiToIkiwikiConversion/) that HTML::WikiConverter needed for your conversion was not up-to-date on Debian I have now done an unofficial package, including your proposed Markdown patches, apt-get'able at <pre>deb http://debian.jones.dk/ sid wikitools</pre>
>> -- [[JonasSmedegaard]]


>>I see the "We will post the scripts ...." was committed about a year ago.  A current site search for "Moin" does not turn them up.  Any chance of an appearance in the near (end of year) future?
>>
>> -- [[MichaelRasmussen]]

----

# LaTeX support?

Moved to [[todo/latex]] --[[Joey]]

----

# Using with CVS?

Moved to a [[todo_item|todo/CVS_backend]]. --[[JoshTriplett]]

----

# Show differences before saving page?

Moved to the existing [[todo_item|todo/preview_changes]]. --[[JoshTriplett]]

----

# Max submit size?

Any setting for limiting how many kilobytes can be submitted via the "edit" form?
-- [[JeremyReed]]

>>> See [[todo/fileupload]] for an idea on limiting page size. --[[Joey]]

----

# Access Keys

Would anyone else find this a valuable addition.  In oddmuse and instiki (the only other
wiki engines I am currently using, the edit, home, and submit link tags have an
accesskey attribute.  I find it nice not to have to resort to the mouse for those
actions.  However, it may not be something everyone appreciates.  Any thoughts?
--[Mazirian](http://mazirian.com)

> Maybe, although it would need to take the critisism at
> <http://www.cs.tut.fi/~jkorpela/forms/accesskey.html> into account.

>> Thank you for that link. Given that the edit link is the first thing you tab to
>> in the current layout, I guess it isn't all that necessary. I have had a 
>> a user complaint recently that Alt-e in oddmuse was overriding his access
>> to the browser menu.

----

# Editing the style sheet.

It would be nice to be able to edit the stylesheet by means of the cgi.  Or is this possible?  I wasn't able to achieve it.
Ok, that's my last 2 cents for a while. --[Mazirian](http://mazirian.com)

> I don't support editing it, but if/when ikiwiki gets [[todo/fileupload]] support,
> it'll be possible to upload a style sheet. (If .css is in the allowed
> extensions list.. no idea how safe that would be, a style sheet is
> probably a great place to put XSS attacks and evil javascript that would
> be filtered out of any regular page in ikiwiki). --[[Joey]]

>> I hadn't thought of that at all.  It's a common feature and one I've
>> relied on safely, because the wikis I am maintaining at the moment
>> are all private and restricted to trusted users. Given that the whole
>> point of ikiwiki is to be able to access and edit via the shell as
>> well as the web, I suppose the features doesn't add a lot.  By the
>> way, the w3m mode is brilliant.  I haven't tried it yet, but the idea
>> is great.

----

# Should not create an existing page

This might be a bug, but will discuss it here first.
Clicking on an old "?" or going to a create link but new Markdown content exists, should not go into "create" mode, but should do a regular "edit".

> I belive that currently it does a redirect to the new static web page.
> At least that's the intent of the code. --[[Joey]]

>> Try at your site: `?page=discussion&from=index&do=create`
>> It brings up an empty textarea to start a new webpage -- even though it already exists here. --reed

>>> Ah, right. Notice that the resulting form allows saving the page as
>>> discussion, or users/discussion, but not index/discussion, since this
>>> page already exists. If all the pages existed, it would do the redirect
>>> thing. --[[Joey]]

----

# User database tools?

Any tool to view user database?

Any tool to edit the user database?

> No, but it's fairly easy to write such tools in perl. For example, to
> list all users in the user database:

        joey@kodama:~/src/joeywiki/.ikiwiki>perl -le 'use Storable; my $userinfo=Storable::retrieve("userdb"); print $_ foreach keys %$userinfo'         
        http://joey.kitenet.net/
        foo

> To list each user's email address:

        joey@kodama:~/src/joeywiki/.ikiwiki>perl -le 'use Storable; my $userinfo=Storable::retrieve("userdb"); print $userinfo->{$_}->{email} foreach keys %$userinfo'
        
        joey@kitenet.net

> Editing is simply a matter of changing values and calling Storable::store().
> I've not written actual utilities to do this yet because I've only needed
> to do it rarely, and the data I've wanted has been different each time.
> --[[Joey]]

>> Thanks for these examples -- I have been using them. I don't know the
>> Storable yet. Can someone share an example of removing a user? (I now
>> setup account\_creation\_password and I have some spammer with different
>> login names that I have banned that I might as well remove from the
>> userdb.)

>>> Let's see, you could do something like this:
>>>     perl -le 'use Storable; my $userinfo=Storable::retrieve("userdb"); delete $$userinfo{"joey"}; Storable::lock_store($userinfo, "userdb")'
>>> I suppose I should stop being lame and create a command line tool wrapping up these operations.. --[[Joey]]

----

# Spaces in WikiLinks?

Hello Joey,

I've just switched from ikiwiki 2.0 to ikiwiki 2.2 and I'm really surprised
that I can't use the spaces in WikiLinks. Could you please tell me why the spaces
aren't allowed in WikiLinks now?

My best regards,

--[[Paweł|ptecza]]

> See [[bugs/Spaces_in_link_text_for_ikiwiki_links]]

----

# Build in OpenSolaris?

Moved to [[bugs/build_in_opensolaris]] --[[Joey]]

----

# Various ways to use Subversion with ikiwiki

I'm playing around with various ways that I can use subversion with ikiwiki.

* Is it possible to have ikiwiki point to a subversion repository which is on a different server?  The basic checkin/checkout functionality seems to work but there doesn't seem to be any way to make the post-commit hook work for a non-local server?

> This is difficult to do since ikiwiki's post-commit wrapper expects to
> run on a machine that contains both the svn repository and the .ikiwiki
> state directory. However, with recent versions of ikiwiki, you can get
> away without running the post-commit wrapper on commit, and all you lose
> is the ability to send commit notification emails.

> (And now that [[recentchanges]] includes rss, you can just subscribe to
> that, no need to worry about commit notification emails anymore.)

* Is it possible / sensible to have ikiwiki share a subversion repository with other data (either completely unrelated files or another ikiwiki instance)?  This works in part but again the post-commit hook seems problematic.

--[[AdamShand]]

> Sure, see ikiwiki's subversion repository for example of non-wiki files
> in the same repo. If you have two wikis in one repository, you will need
> to write a post-commit script that calls the post-commit wrappers for each
> wiki.

----

# Regex for Valid Characters in Filenames

I'm sure that this is documented somewhere but I've ransacked the wiki and I can't find it.  :-(  What are the allowed characters in an ikiwiki page name?  I'm writing a simple script to make updating my blog easier and need to filter invalid characters (so far I've found that # and , aren't allowed ;-)).  Thanks for any pointers.  -- [[AdamShand]]

> The default `wiki_file_regexp` matches filenames containing only
> `[-[:alnum:]_.:/+]`
> 
> The IkiWiki::titlepage() function will convert freeform text to a valid
> page name. See [[todo/should_use_a_standard_encoding_for_utf_chars_in_filenames]]
> for an example. --[[Joey]]

>> Perfect, thanks! 
>>
>> In the end I decided that I didn't need any special characters in filenames and replaced everything but alphanumeric characters with underscores.  In addition to replacing bad characters I also collapse multiple underscores into a single one, and strip off trailing and leading underscores to make tidy filenames.  If it's useful to anybody else here's a sed example:
>>
>>     # echo "++ Bad: ~@#$%^&*()_=}{[];,? Iki: +_-:./ Num: 65.5 ++" | sed -e 's/[^A-Za-z0-9_]/_/g' -e 's/__*/_/g' -e 's/^_//g' -e 's/_$//g'
>>     Bad_Iki_Num_65_5
>>
>>--[[AdamShand]]