added.
This would also be useful for autoindex, as suggested on
-[[plugins/autoindex/discussion]]. I'd also like to use it for
-[[plugins/contrib/album]].
+[[plugins/autoindex/discussion]] and [[!debbug 544322]]. I'd also like
+to use it for [[plugins/contrib/album]].
It could also be used for an [[todo/alias_directive]].
--[[smcv]]
+> All [[merged|done]] --[[Joey]]
+
--------------------------
[[!template id=gitbranch branch=smcv/ready/transient author="[[smcv]]"]]
[[!tag patch]]
+Related branches:
+
+* `ready/tag-test`: an extra regression test for tags
+ > merged --[[Joey]]
+* either `transient-relative` or `transient-relative-api`: avoid using `Cwd`
+ on initialization
+ > merged the latter --[[Joey]]
+* `ready/transient-aggregate`: use for aggregate
+ > merged --[[Joey]]
+* `ready/transient-autoindex`: optionally use for autoindex,
+ which is [[!debbug 544322]] (includes autoindex-autofile from
+ [[todo/autoindex should use add__95__autofile]])
+ > merged. I do note that this interacts badly with ikiwiki-hosting's
+ > backup/restore/branch handling, since that does not back up the
+ > transientdir by default, and so autoindex will not recreate the
+ > "deleted" pages. I'll probably have to make it back up the transientdir
+ > too. --[[Joey]]
+* `ready/transient-recentchanges`: use for recentchanges
+ > merged --[[Joey]]
+* `ready/transient-tag`: optionally use for tag (includes tag-test)
+ > merged --[[Joey]]
+
I think this branch is now enough to be useful. It adds the following:
If the `transient` plugin is loaded, `$srcdir/.ikiwiki/transient` is added
I erred on the side of more plugins; I think it's "on the edge of the core",
like goto.
-Pages with the default extension in the transient underlay are automatically
+Pages in the transient underlay are automatically
deleted if a page of the same name is created in the srcdir (or an underlay
closer to the srcdir in stacking order).
+With the additional `ready/transient-tag` branch,
`tag` enables `transient`, and if `tag_autocreate_commit` is set to 0
(default 1), autocreated tags are written to the transient underlay.
There is a regression test.
+With the additional `transient-autoindex` branch,
`autoindex` uses autofiles. It also enables `transient`, and if
`autoindex_commit` is set to 0 (default 1), autoindexes are written to
-the transient underlay. There is a regression test.
+the transient underlay. There is a regression test. However, this branch
+is blocked by working out what the desired behaviour is, on
+[[todo/autoindex_should_use_add__95__autofile]].
> I wonder why this needs to be configurable? I suppose that gets back to
> whether it makes sense to check these files in or not. The benefits of
> commit clutter is really worth it.
>> According to the last section of
->> [[todo/auto-create_tag_pages_according_to_a_template]], chrysn and
+>> [[todo/auto-create_tag_pages_according_to_a_template]], [[chrysn]] and
>> Eric both feel rather strongly that it should be possible to
->> not commit any tags. I made it configurable because, as you point out,
+>> not commit any tags; in [[plugins/autoindex/discussion]],
+>> lollipopman and [[JoeRayhawk]] both requested the same for autoindex.
+>> I made it configurable because, as you point out,
>> there are also reasons why it makes sense to check these
>> automatically-created files in. I'm neutral on this, personally.
>>
>> If this is a point of contention, would you accept a branch that
>> just adds `transient` and uses it for [[plugins/recentchanges]],
->> which aren't checked in and never have been?
+>> which aren't checked in and never have been? I've split the
+>> branch up in the hope that *some* of it can get merged.
+>>
+>>> I will be happy to merge transient-recentchanges when it's ready.
+>>> I see no obstacle to merging transient-tag either, and am not
+>>> really against using it for autoindex or aggregate either
+>>> once they get completed.
+>>> I just wanted to think through why configurability is needed.
+>>> --[[Joey]]
>>
>> One potentially relevant point is that configuration clutter only
>> affects the site admin whereas commit clutter is part of the whole
> Anyway, the configurability
> appears subtly broken; the default is only 1 if a new setup file is
-> generated. With an existing setup file, the 'default' values in
+> generated. (Correction: It was not even the default then --[[Joey]])
+> With an existing setup file, the 'default' values in
> `getsetup` don't take effect, so it will default to undef, which
> is treated the same as 0. --[[Joey]]
->> Noted. I assume a `checkconfig` hook is the way to fix this. --s
+>> Fixed in the branches, hopefully. (How disruptive would it be to have
+>> defaults take effect whenever the setup file doesn't set a value, btw?
+>> It seems pretty astonishing to have them work as they do at the moment.) --s
+
+>>> Well, note that default is not actually a documented field in
+>>> getsetup hooks at all! (It is used in IkiWiki.pm's own `getsetup()`, and
+>>> the concept may have leaked out into one or two plugins (comments,
+>>> transient)).
+>>>
+>>> Running getsetup at plugin load time is something I have considered
+>>> doing. It would simplify some checkconfig hooks that just set hardcoded
+>>> defaults. Although since dying is part of the getsetup hook's API, it
+>>> could be problimaric.
+>>> --[[Joey]]
autoindex ignores pages in the transient underlay when deciding whether
to generate an index.
-New recent changes go in the transient underlay; I tested this manually.
+With the additional `ready/transient-recentchanges` branch, new recent
+changes go in the transient underlay; I tested this manually.
Not done yet (in that branch, at least):
>> I'll try to work out which of the checks are required for security
>> and which are just nice-to-have, but I'd appreciate any pointers
>> you could give. Note that my branch wasn't meant to enable either
- >> by default. --[[smcv]]
+ >> by default, and now hopefully doesn't. --[[smcv]]
+
+ >>> Opened a new bug for this, [[bugs/removal_of_transient_pages]]
+ >>> --[[Joey]]
* Transient tags that don't match any pages aren't deleted: I'm not sure
that that's a good idea anyway, though. Similarly, transient autoindexes
transient underlay too (they'll naturally migrate over time). I haven't
tested this yet, it's just a proof-of-concept.
+ > Now renamed to `ready/transient-aggregate`; it does seem to work fine.
+ > --s
+
> I can confirm that the behavior of autoindex, at least, is excellent.
> Haven't tried tag. Joey, can you merge transient and autoindex? --JoeRayhawk
>> Here are some other things I'd like to think about first: --[[Joey]]
>>
>> * There's a FIXME in autoindex.
-
->>> Right, the extra logic for preventing autoindex pages from being
->>> re-created. I've started on regression tests for this, but I'll
->>> need to rummage through the commit log for the exact requirements
->>> and the reasoning behind them, so it could take a while. --s
+>>
+>> > Right, the extra logic for preventing autoindex pages from being
+>> > re-created. This is taking a while, so I'm going to leave out the
+>> > autoindex part for the moment. The FIXME is only relevant
+>> > because I tried to solve
+>> > [[todo/autoindex should use add__95__autofile]] first, but
+>> > strictly speaking, that's an orthogonal change. --s
>> * Suggest making recentchanges unlink the transient page
>> first, and only unlink from the old location if it wasn't
>> in the transient location. Ok, it only saves 1 syscall :)
+>>
+>> > Is an unlink() really that expensive? But, OK, fixed in the
+>> > `ready/transient-recentchanges` branch. --s
->>> Is an unlink() really that expensive? But, OK. --s
+>> >> It's not, but it's easy. :) --[[Joey]]
>> * Similarly it's a bit worrying for performance that it
>> needs to pull in and use `Cwd` on every ikiwiki startup now.
>> I really don't see the need; `wikistatedir` should
>> mostly be absolute, and ikiwiki should not chdir in ways
>> that break it anyway.
-
->>> The reason to make it absolute is that relative underlays
->>> are interpreted as relative to the base underlay directory,
->>> not the cwd. I could conditionalize the use of `Cwd` on
->>> `wikistatedir` not being absolute, though, which in practice
->>> will only happen for unusual deployments (like the docwiki);
->>> or I could add some mechanism by which you can add a
->>> cwd-relative underlay? --s
+>>
+>> > The reason to make it absolute is that relative underlays
+>> > are interpreted as relative to the base underlay directory,
+>> > not the cwd, by `add_underlay`.
+>> >
+>> > The updated `ready/transient-only` branch only loads `Cwd` if
+>> > the path is relative; an extra commit on branch
+>> > `smcv/transient-relative` goes behind `add_underlay`'s
+>> > back to allow use of a cwd-relative underlay. Which direction
+>> > would you prefer?
+>> >
+>> > I note in passing that [[plugins/autoindex]] and `IkiWiki::Render`
+>> > both need to use `Cwd` and `File::Find` on every refresh, so
+>> > there's only any point in avoiding `Cwd` for runs that don't
+>> > actually refresh, like simple uses of the CGI. --s
+
+>> >> Oh, right, I'd forgotten about the horrificness of File::Find
+>> >> that required a chdir for security. Ugh. Can we just avoid
+>> >> it for those simple cases then? (demand-calculate wikistatedir)
+>> >> --[[Joey]]
+
+>> >>> The reason that transientdir needs to be absolute is that it's
+>> >>> added as an underlay.
+>> >>>
+>> >>> We could avoid using `Cwd` by taking the extra commit from either
+>> >>> `smcv/transient-relative` or `smcv/transient-relative-api`;
+>> >>> your choice. I'd personally go for the latter.
+>> >>>
+>> >>> According to git grep, [[plugins/po]] already wants to look at
+>> >>> the underlaydirs in its checkconfig hook, so I don't think
+>> >>> delaying calculation of the underlaydir is viable. (I also noticed
+>> >>> a bug,
+>> >>> [[bugs/po:_might_not_add_translated_versions_of_all_underlays]].)
+>> >>>
+>> >>> `underlaydirs` certainly needs to have been calculated by the
+>> >>> time `refresh` hooks finish, so `find_src_files` can use it. --s
>> * Unsure about the use of `default_pageext` in the `change`
>> hook. Is everything in the transientdir really going
>> to use that pageext? Would it be better to look up the
>> complete source filename?
-
->>> At the moment everything in the transientdir will either
->>> have the `default_pageext` or be internal, although I
->>> did wonder whether to make [[plugins/contrib/album]]
->>> viewer pages optionally be `html`, for better performance
->>> when there's a very large number of photos.
->>>
->>> A more thorough garbage-collection mechanism would be to
->>> use File::Find on the transient directory; I'll get there
->>> eventually. --s
+>>
+>> > I've updated `ready/transient` to do a more thorough GC by
+>> > using File::Find on the transient directory. This does
+>> > require `File::Find` and `Cwd`, but only when pages change,
+>> > and `refresh` loads both of those in that situation anyway.
+>> >
+>> > At the moment everything in the transientdir will either
+>> > have the `default_pageext` or be internal, although I
+>> > did wonder whether to make [[plugins/contrib/album]]
+>> > viewer pages optionally be `html`, for better performance
+>> > when there's a very large number of photos. --s
+
+>> >> Oh, ugh, more File::Find... Couldn't it just assume that the
+>> >> transient page has the same extension as its replacement?
+>> >> --[[Joey]]
+
+>> >>> Good idea, that'll be true for web edits at least.
+>> >>> Commit added. --s
--------------------------