Only unlink excess recentchanges from srcdir if not in transientdir

[git.ikiwiki.info.git] / doc / todo / transient_pages.mdwn

diff --git a/doc/todo/transient_pages.mdwn b/doc/todo/transient_pages.mdwn
index ea1ff6a8fc379f60d43f764c6bd426cf749b8fcd..1a35dddb10c60bbe978ceeb6cbbf31435b12b63c 100644 (file)
--- a/doc/todo/transient_pages.mdwn
+++ b/doc/todo/transient_pages.mdwn
@@ -15,13 +15,15 @@ This would also be useful for autoindex, as suggested on
 [[plugins/autoindex/discussion]]. I'd also like to use it for
 [[plugins/contrib/album]].
 
 [[plugins/autoindex/discussion]]. I'd also like to use it for
 [[plugins/contrib/album]].
 

-One refinement I'd suggest is that if the transient page is edited,
-its transient contents are evaluated and used as the initial
-content for the edit box; after that, it'd become a static page. --[[smcv]]
+It could also be used for an [[todo/alias_directive]].
+
+--[[smcv]]

 
 --------------------------
 
 [[!template id=gitbranch branch=smcv/ready/transient author="[[smcv]]"]]
 
 --------------------------
 
 [[!template id=gitbranch branch=smcv/ready/transient author="[[smcv]]"]]

+[[!template id=gitbranch branch=smcv/ready/transient-recentchanges author="[[smcv]]"]]
+[[!template id=gitbranch branch=smcv/ready/transient-tag author="[[smcv]]"]]

 [[!tag patch]]
 
 I think this branch is now enough to be useful. It adds the following:
 [[!tag patch]]
 
 I think this branch is now enough to be useful. It adds the following:


@@ -31,17 +33,21 @@ as an underlay. I'm not sure whether this should be a plugin or core, so
 I erred on the side of more plugins; I think it's "on the edge of the core",
 like goto.
 
 I erred on the side of more plugins; I think it's "on the edge of the core",
 like goto.
 

-Pages with the default extension in the transient underlay are automatically
+Pages in the transient underlay are automatically

 deleted if a page of the same name is created in the srcdir (or an underlay
 closer to the srcdir in stacking order).
 
 deleted if a page of the same name is created in the srcdir (or an underlay
 closer to the srcdir in stacking order).
 

+With the additional `ready/transient-tag` branch,

 `tag` enables `transient`, and if `tag_autocreate_commit` is set to 0
 (default 1), autocreated tags are written to the transient underlay.
 There is a regression test.
 
 `tag` enables `transient`, and if `tag_autocreate_commit` is set to 0
 (default 1), autocreated tags are written to the transient underlay.
 There is a regression test.
 

+With the additional `transient-autoindex` branch,

 `autoindex` uses autofiles. It also enables `transient`, and if
 `autoindex_commit` is set to 0 (default 1), autoindexes are written to
 `autoindex` uses autofiles. It also enables `transient`, and if
 `autoindex_commit` is set to 0 (default 1), autoindexes are written to

-the transient underlay. There is a regression test.
+the transient underlay. There is a regression test. However, this branch
+is blocked by working out what the desired behaviour is, on
+[[todo/autoindex_should_use_add__95__autofile]].

 
 > I wonder why this needs to be configurable? I suppose that gets back to
 > whether it makes sense to check these files in or not. The benefits of 
 
 > I wonder why this needs to be configurable? I suppose that gets back to
 > whether it makes sense to check these files in or not. The benefits of 


@@ -52,22 +58,69 @@ the transient underlay. There is a regression test.
 >   but does it make sense to make that wart larger?
 > * You can know you can build the same site with nothing missing
 >   even if you don't there enable autoindex or whatever. (Edge case.)
 >   but does it make sense to make that wart larger?
 > * You can know you can build the same site with nothing missing
 >   even if you don't there enable autoindex or whatever. (Edge case.)

-> 
+
+>> I'm not sure that that's a huge wart; you can always "edit by
+>> overwriting". If you're running a local clone of the wiki on your laptop
+>> or whatever, you have the underlays already, and can copy from there.
+>> Tag and autoindex pages have rather simple source code anyway. --s
+

 > The benefit of using transient pages seems to just be avoiding commit
 > clutter? For files that are never committed, transient pages are a clear
 > win, but I wonder if adding configuration clutter just to avoid some 
 > commit clutter is really worth it.
 > The benefit of using transient pages seems to just be avoiding commit
 > clutter? For files that are never committed, transient pages are a clear
 > win, but I wonder if adding configuration clutter just to avoid some 
 > commit clutter is really worth it.

-> 
+
+>> According to the last section of
+>> [[todo/auto-create_tag_pages_according_to_a_template]], [[chrysn]] and
+>> Eric both feel rather strongly that it should be possible to
+>> not commit any tags; in [[plugins/autoindex/discussion]],
+>> lollipopman and [[JoeRayhawk]] both requested the same for autoindex.
+>> I made it configurable because, as you point out,
+>> there are also reasons why it makes sense to check these
+>> automatically-created files in. I'm neutral on this, personally.
+>>
+>> If this is a point of contention, would you accept a branch that
+>> just adds `transient` and uses it for [[plugins/recentchanges]],
+>> which aren't checked in and never have been? I've split the
+>> branch up in the hope that *some* of it can get merged.
+>>
+>>> I will be happy to merge transient-recentchanges when it's ready.
+>>> I see no obstacle to merging transient-tag either, and am not
+>>> really against using it for autoindex or aggregate either
+>>> once they get completed.
+>>> I just wanted to think through why configurability is needed.
+>>> --[[Joey]]
+>>
+>> One potentially relevant point is that configuration clutter only
+>> affects the site admin whereas commit clutter is part of the whole
+>> wiki's history. --[[smcv]]
+

 > Anyway, the configurability
 > appears subtly broken; the default is only 1 if a new setup file is
 > Anyway, the configurability
 > appears subtly broken; the default is only 1 if a new setup file is

-> generated. With an existing setup file, the 'default' values in
+> generated. (Correction: It was not even the default then --[[Joey]])
+> With an existing setup file, the 'default' values in

 > `getsetup` don't take effect, so it will default to undef, which
 > is treated the same as 0. --[[Joey]]
 
 > `getsetup` don't take effect, so it will default to undef, which
 > is treated the same as 0. --[[Joey]]
 

+>> Fixed in the branches, hopefully. (How disruptive would it be to have
+>> defaults take effect whenever the setup file doesn't set a value, btw?
+>> It seems pretty astonishing to have them work as they do at the moment.) --s
+
+>>> Well, note that default is not actually a documented field in
+>>> getsetup hooks at all! (It is used in IkiWiki.pm's own `getsetup()`, and
+>>> the concept may have leaked out into one or two plugins (comments,
+>>> transient)).
+>>> 
+>>> Running getsetup at plugin load time is something I have considered
+>>> doing. It would simplify some checkconfig hooks that just set hardcoded 
+>>> defaults. Although since dying is part of the getsetup hook's API, it
+>>> could be problimaric.
+>>> --[[Joey]]
+

 autoindex ignores pages in the transient underlay when deciding whether
 to generate an index.
 
 autoindex ignores pages in the transient underlay when deciding whether
 to generate an index.
 

-New recent changes go in the transient underlay; I tested this manually.
+With the additional `ready/transient-recentchanges` branch, new recent
+changes go in the transient underlay; I tested this manually.

 
 Not done yet (in that branch, at least):
 
 
 Not done yet (in that branch, at least):
 


@@ -78,12 +131,19 @@ Not done yet (in that branch, at least):
   > Hmm, I'd at least want that to be dealt with before this was used
   > by default for autoindex or tag. --[[Joey]]
 
   > Hmm, I'd at least want that to be dealt with before this was used
   > by default for autoindex or tag. --[[Joey]]
 

+  >> I'll try to work out which of the checks are required for security
+  >> and which are just nice-to-have, but I'd appreciate any pointers
+  >> you could give. Note that my branch wasn't meant to enable either
+  >> by default, and now hopefully doesn't. --[[smcv]]
+

 * Transient tags that don't match any pages aren't deleted: I'm not sure
   that that's a good idea anyway, though. Similarly, transient autoindexes
   of directories that become empty aren't deleted.
 
   > Doesn't seem necessary, or really desirable to do that. --[[Joey]]
 
 * Transient tags that don't match any pages aren't deleted: I'm not sure
   that that's a good idea anyway, though. Similarly, transient autoindexes
   of directories that become empty aren't deleted.
 
   > Doesn't seem necessary, or really desirable to do that. --[[Joey]]
 

+  >> Good, that was my inclination too. --s
+

 * In my `untested/transient` branch, new aggregated files go in the
   transient underlay too (they'll naturally migrate over time). I haven't
   tested this yet, it's just a proof-of-concept.
 * In my `untested/transient` branch, new aggregated files go in the
   transient underlay too (they'll naturally migrate over time). I haven't
   tested this yet, it's just a proof-of-concept.


@@ -94,18 +154,68 @@ Not done yet (in that branch, at least):
 >> Here are some other things I'd like to think about first: --[[Joey]] 
 >>
 >> * There's a FIXME in autoindex.
 >> Here are some other things I'd like to think about first: --[[Joey]] 
 >>
 >> * There's a FIXME in autoindex.

+>>
+>>     > Right, the extra logic for preventing autoindex pages from being
+>>     > re-created. This is taking a while, so I'm going to leave out the
+>>     > autoindex part for the moment. The FIXME is only relevant
+>>     > because I tried to solve
+>>     > [[todo/autoindex should use add__95__autofile]] first, but
+>>     > strictly speaking, that's an orthogonal change. --s
+

 >> * Suggest making recentchanges unlink the transient page
 >>   first, and only unlink from the old location if it wasn't
 >>   in the transient location. Ok, it only saves 1 syscall :)
 >> * Suggest making recentchanges unlink the transient page
 >>   first, and only unlink from the old location if it wasn't
 >>   in the transient location. Ok, it only saves 1 syscall :)

+>>
+>>   > Is an unlink() really that expensive? But, OK, fixed in the
+>>   > `ready/transient-recentchanges` branch. --s
+
+>>   >> It's not, but it's easy. :) --[[Joey]]
+

 >> * Similarly it's a bit worrying for performance that it
 >>   needs to pull in and use `Cwd` on every ikiwiki startup now.
 >>   I really don't see the need; `wikistatedir` should
 >>   mostly be absolute, and ikiwiki should not chdir in ways
 >>   that break it anyway.
 >> * Similarly it's a bit worrying for performance that it
 >>   needs to pull in and use `Cwd` on every ikiwiki startup now.
 >>   I really don't see the need; `wikistatedir` should
 >>   mostly be absolute, and ikiwiki should not chdir in ways
 >>   that break it anyway.

+>>
+>>   > The reason to make it absolute is that relative underlays
+>>   > are interpreted as relative to the base underlay directory,
+>>   > not the cwd, by `add_underlay`.
+>>   >
+>>   > The updated `ready/transient-only` branch only loads `Cwd` if
+>>   > the path is relative; an extra commit on branch
+>>   > `smcv/transient-relative` goes behind `add_underlay`'s
+>>   > back to allow use of a cwd-relative underlay. Which direction
+>>   > would you prefer?
+>>   >
+>>   > I note in passing that [[plugins/autoindex]] and `IkiWiki::Render`
+>>   > both need to use `Cwd` and `File::Find` on every refresh, so
+>>   > there's only any point in avoiding `Cwd` for runs that don't
+>>   > actually refresh, like simple uses of the CGI. --s
+
+>>   >> Oh, right, I'd forgotten about the horrificness of File::Find
+>>   >> that required a chdir for security. Ugh. Can we just avoid
+>>   >> it for those simple cases then? (demand-calculate wikistatedir)
+>>   >> --[[Joey]] 
+

 >> * Unsure about the use of `default_pageext` in the `change`
 >>   hook. Is everything in the transientdir really going
 >>   to use that pageext? Would it be better to look up the
 >>   complete source filename?
 >> * Unsure about the use of `default_pageext` in the `change`
 >>   hook. Is everything in the transientdir really going
 >>   to use that pageext? Would it be better to look up the
 >>   complete source filename?

+>>
+>>   > I've updated `ready/transient` to do a more thorough GC by
+>>   > using File::Find on the transient directory. This does
+>>   > require `File::Find` and `Cwd`, but only when pages change,
+>>   > and `refresh` loads both of those in that situation anyway.
+>>   >
+>>   > At the moment everything in the transientdir will either
+>>   > have the `default_pageext` or be internal, although I
+>>   > did wonder whether to make [[plugins/contrib/album]]
+>>   > viewer pages optionally be `html`, for better performance
+>>   > when there's a very large number of photos. --s
+
+>>   >> Oh, ugh, more File::Find... Couldn't it just assume that the
+>>   >> transient page has the same extension as its replacement?
+>>   >> --[[Joey]]

 
 --------------------------
 
 
 --------------------------