changelog, close bug

[git.ikiwiki.info.git] / doc / tips / distributed_wikis.mdwn

diff --git a/doc/tips/distributed_wikis.mdwn b/doc/tips/distributed_wikis.mdwn
index 75a2368b11bc88101e062793c08472040a4a2c28..2547a2e9fe510bdbb8ab22a06f847d1a47aaac47 100644 (file)
--- a/doc/tips/distributed_wikis.mdwn
+++ b/doc/tips/distributed_wikis.mdwn
@@ -5,52 +5,69 @@ git, let's explore some possibilities for distributed wikis.
 
 [[!toc levels=2]]
 
 
 [[!toc levels=2]]
 

-## a wiki mirror
+## Overview

 
 

-The simplest possibility is setting up a mirror. If a wiki exposes its git
-repository and has the [[plugins/pinger]] plugin enabled, then anyone can
-set up a mirror that will automatically be kept up-to-date with the origin
-wiki. Just clone the git repo, configure ikiwiki to use it, enable the
-[[plugins/pingee]] plugin in your configuration, and edit the origin wiki,
-adding a ping directive for your mirror:
+There are several possible level of decentralisation:

 
 

-       \[[!ping from="http://thewiki.com/"
-       to="http://mymirror.com/ikiwiki.cgi?do=ping"]]
+ 0. [[default setup|rcs/git]], no decentralisation
+ 1. [[a simple HTML mirror|tips/Git_repository_and_web_server_on_different_hosts/]]
+ 2. [[separate ikiwiki and git servers|tips/Hosting_Ikiwiki_and_master_git_repository_on_different_machines]]
+ 3. separate `srcdir`, still requires a central bare repo - uses [[plugins/pinger]]
+ 4. completely distinct ikiwiki installs, synchronised with [[plugins/contrib/gitpush]]

 
 

-The "from" parameter needs to be the url to the origin wiki. The "to" parameter
-is the url to ping on your mirror.
+Here's a graphic overview of those:

 
 

-Now whenever the main wiki is edited, it will ping your mirror, which will
-pull the changes from "origin" using git, and update itself. It could, in
-turn ping another mirror, etc.
+### Default setup - one central server

 
 

-And if someone edits a page on your mirror, it will "git push origin",
-committing the changes back to the origin git repository, and updating the
-origin mirror. Assuming you can push to that git repository. If you can't,
-and you want a mirror, and not a branch, you should disable web edits on
-your mirror. (You could also point the cgiurl for your mirror at the origin
-wiki.)
+[[!img rcs/git/wiki_edit_flow.svg size=400x]]

 
 

-## branching a wiki
+In the default setup, all the resources are stored on the central
+servers. Users can still clone and edit the git repo by hand and
+contribute by git, but otherwise all the changes happen on a single
+web interface. This basic setup is best described in [[rcs/git]].

 
 

-It follows that setting up a branch of a wiki is just like a mirror, except
-we don't want it to push changes back to the origin. The easy way to
-accomplish this is to clone the origin git repository using a readonly
-protocol (ie, "git://"). Then you can't push to it.
+### Separate webserver and git repository

 
 

-If a page on your branch is modified and other modifications are made to
-the same page in the origin, a conflict might occur when that change is
-pulled in. How well will this be dealt with and how to resolve it? I think
-that the conflict markers will just appear on the page as it's rendered in
-the wiki, and if you could even resolve the conflict using the web
-interface. Not 100% sure as I've not gotten into this situation yet.
+[[!img tips/Git_repository_and_web_server_on_different_hosts/separate-webserver.svg size=400x]]

 
 

---[[Joey]]
+This is the configuration described in
+[[tips/Git_repository_and_web_server_on_different_hosts]]. The webserver part
+hosts the HTML files, the ikiwiki [[cgi]] but everything else is on
+the git server.
+
+### Separate webserver and git repository, the git srcdir being hosted on the webserver
+
+[[!img Hosting_Ikiwiki_and_master_git_repository_on_different_machines/separate-web-git-servers.svg size=400x]]
+
+This is the configuration described in
+[[tips/Hosting_Ikiwiki_and_master_git_repository_on_different_machines]]. One server hosts the web server (and the [[Ikiwiki cgi|cgi]]) and the git source dir; a second server hosts the git bare repository. This can be used when you have very limited access to the git server.
+
+### Decentralised pinger setup
+
+[[!img ping-setup.svg size=400x]]
+
+In this configuration, the mirrors all have their own `srcdir`, but
+still need to push and pull from the same central bare git repo. The
+[[plugins/pinger]] plugin is used to ping the mirrors from the central
+server on edits.
+
+Step by step setup instructions for this are detailed below.

 
 

-## Practical example
+### Fully decentralised setup
+
+[[!img decentralized_wikis.svg size=400x]]
+
+In this configuration, each wiki is fully independent and pushes its
+changes to other wikis using the [[plugins/contrib/gitpush]] plugin.
+
+## Step by step setup instructions
+
+The first two ways of setting up ikiwiki are better described in [[setup]] or [[tips/Git_repository_and_web_server_on_different_hosts]]. The remainder of this page describes the latter two more complex distributed setups. 

 
 Say you have a friend that has already configured a shiny ikiwiki site, and you want to help by creating a mirror. You still need to figure out how to install ikiwiki and everything, hopefully this section will help you with that.
 
 
 Say you have a friend that has already configured a shiny ikiwiki site, and you want to help by creating a mirror. You still need to figure out how to install ikiwiki and everything, hopefully this section will help you with that.
 

+Note that parts of the following documentation duplicate instructions from [[setup]], [[setup/byhand]], [[rcs/git]] and [[tips/laptop_wiki_with_git]].
+

 ### Installing ikiwiki
 
 You need to install the ikiwiki package for the mirror to work. You can use ikiwiki to publish the actual HTML pages elsewhere if you don't plan on letting people edit the wiki, but generally you want the package to be installed on the webserver for editing to work.
 ### Installing ikiwiki
 
 You need to install the ikiwiki package for the mirror to work. You can use ikiwiki to publish the actual HTML pages elsewhere if you don't plan on letting people edit the wiki, but generally you want the package to be installed on the webserver for editing to work.


@@ -96,14 +113,14 @@ You should already be able to make a plain HTML rendering of the wiki:
 You will also need a webserver to serve the content in the `destdir`
 defined above. We assume you will configure a virtual host named `mirror.example.com`. Here are some examples on how to do those, see [[!iki setup]] and [[!iki tips/dot_cgi]] for complete documentation.
 
 You will also need a webserver to serve the content in the `destdir`
 defined above. We assume you will configure a virtual host named `mirror.example.com`. Here are some examples on how to do those, see [[!iki setup]] and [[!iki tips/dot_cgi]] for complete documentation.
 

-Note that this will also configure CGI so that people can edit the wiki. Note that this configuration may involve timeouts if the main site is down.
+Note that this will also configure CGI so that people can edit the wiki. Note that this configuration may involve timeouts if the main site is down, as ikiwiki will attempt to push to the central git repository at every change.

 
 #### Apache configuration
 
     <VirtualHost *:80>
 
 #### Apache configuration
 
     <VirtualHost *:80>

-        ServerName reseaulibre.example.com:80
-        DocumentRoot /var/www/reseaulibre.example.com
-        <Directory /var/www/reseaulibre.example.com>
+        ServerName mirror.example.com:80
+        DocumentRoot /var/www/mirror.example.com
+        <Directory /var/www/mirror.example.com>

             Options Indexes MultiViews ExecCGI
             AllowOverride None
             Order allow,deny
             Options Indexes MultiViews ExecCGI
             AllowOverride None
             Order allow,deny


@@ -116,9 +133,9 @@ Note that this will also configure CGI so that people can edit the wiki. Note th
 #### Nginx configuration
 
     server {
 #### Nginx configuration
 
     server {

-        root /var/www/reseaulibre.example.com/;
+        root /var/www/mirror.example.com/;

         index index.html index.htm;
         index index.html index.htm;

-        server_name reseaulibre.example.com;
+        server_name mirror.example.com;

 
         location / {
             try_files $uri $uri/ /index.html;
 
         location / {
             try_files $uri $uri/ /index.html;


@@ -127,7 +144,7 @@ Note that this will also configure CGI so that people can edit the wiki. Note th
             fastcgi_pass  unix:/tmp/fcgi.socket;
             fastcgi_index ikiwiki.cgi;
             fastcgi_param SCRIPT_FILENAME   /var/www/ikiwiki.cgi;
             fastcgi_pass  unix:/tmp/fcgi.socket;
             fastcgi_index ikiwiki.cgi;
             fastcgi_param SCRIPT_FILENAME   /var/www/ikiwiki.cgi;

-            fastcgi_param  DOCUMENT_ROOT      /var/www/reseaulibre.example.com;
+            fastcgi_param  DOCUMENT_ROOT      /var/www/mirror.example.com;

             include /etc/nginx/fastcgi_params;
         }
     }
             include /etc/nginx/fastcgi_params;
         }
     }


@@ -141,20 +158,33 @@ Make this writable:
 
     chmod a+w /tmp/fcgi.socket
 
 
     chmod a+w /tmp/fcgi.socket
 

-### Read-only mirror: done!
+### Enable the pinger functionality

 
 

-At this point, you are done! You can edit your own clone of the wiki, although your changes will not go back to the main site. However, you can always push or pull manually from the `repository` in `~user/source.git` to update the main site.
+At this point, you need to enable the pinger functionality to make sure that changes on the central server propagate to your mirror.

 
 

-### Announcing the mirror
+This assumes a central wiki that exposes its git
+repository and has the [[plugins/pinger]] plugin enabled. Enable the
+[[plugins/pingee]] plugin in your configuration, and edit the origin wiki,
+adding a ping directive for your mirror:

 
 

-Once your mirror works, you can also add it to the list of mirrors. You can ask the mirror where you take it from (and why not, all mirrors) to add it to their setup file. As an example, here's the configuration for the first mirror:
+       \[[!ping from="http://thewiki.com/"
+       to="http://mymirror.com/ikiwiki.cgi?do=ping"]]

 
 

-    mirrorlist:
-      example: https://wiki.example.com/
+The "from" parameter needs to be the url to the origin wiki. The "to" parameter
+is the url to ping on your mirror. This can be done basically in any page.

 
 

-The [[plugins/mirrorlist]] plugin of course needs to be enabled for this to work.
+Now whenever the main wiki is edited, it will ping your mirror, which will
+pull the changes from "origin" using git, and update itself. It could, in
+turn ping another mirror, etc.

 
 

-### Alternative configuration
+And if someone edits a page on your mirror, it will "git push origin",
+committing the changes back to the origin git repository, and updating the
+origin mirror. Assuming you can push to that git repository. If you can't,
+and you want a mirror, and not a branch, you should disable web edits on
+your mirror. (You could also point the cgiurl for your mirror at the origin
+wiki if you do not want to incur that overhead or do not want to, or can't, run a CGI.)
+
+### Fully decentralized configuration

 
 In the above configuration, the master git repository is still on the main site. If that site goes down, there will be delays when editing the wiki mirror. It could also simply fail because it will not be able to push the changes to the master git repo. An alternative is to setup a local bare repository that is synced with the master.
 
 
 In the above configuration, the master git repository is still on the main site. If that site goes down, there will be delays when editing the wiki mirror. It could also simply fail because it will not be able to push the changes to the master git repo. An alternative is to setup a local bare repository that is synced with the master.
 


@@ -175,17 +205,44 @@ The following entries will be different from the above setup file:
     git_wrapper: /home/user/source.git/hooks/post-commit
     git_test_receive_wrapper: /home/user/source.git/hooks/pre-receive
 
     git_wrapper: /home/user/source.git/hooks/post-commit
     git_test_receive_wrapper: /home/user/source.git/hooks/pre-receive
 

-To do this, the mirror needs to push back to the master, again using the gitpush plugin:
+To do this, the mirror needs to push back to the master, using the [[plugins/contrib/gitpush]] plugin:

 
     git_push_to:
     - git://wiki.example.com/
 
 This will ensure that commits done on the mirror will propagate back to the master.
 
 
     git_push_to:
     - git://wiki.example.com/
 
 This will ensure that commits done on the mirror will propagate back to the master.
 

-### Other guides
+## Other ideas
+
+See also:
+
+ * [[setup]]
+ * [[setup/byhand]]
+ * [[rcs/git]]
+ * [[tips/laptop_wiki_with_git]]
+ * [ikiwiki creation notes](http://piny.be/jrayhawk/notes/ikiwiki_creation/)
+
+### Announcing the mirror
+
+Once your mirror works, you can also add it to the list of mirrors. You can ask the mirror where you take it from (and why not, all mirrors) to add it to their setup file. As an example, here's the configuration for the first mirror:
+
+    mirrorlist:
+      example: https://wiki.example.com/
+
+The [[plugins/mirrorlist]] plugin of course needs to be enabled for this to work.

 
 

-Another guide is the [[tips/laptop_wiki_with_git]] guide. To get a
-better understanding of how ikiwiki works, see [[rcs/git]].
+### branching a wiki

 
 

-[This](http://piny.be/jrayhawk/notes/ikiwiki_creation/) may also be of
-use if the above doesn't work.
+It follows that setting up a branch of a wiki is just like the fully decentralised mirror above, except
+we don't want it to push changes back to the origin. The easy way to
+accomplish this is to clone the origin git repository using a readonly
+protocol (ie, "git://"). Then you can't push to it.
+
+If a page on your branch is modified and other modifications are made to
+the same page in the origin, a conflict might occur when that change is
+pulled in. How well will this be dealt with and how to resolve it? I think
+that the conflict markers will just appear on the page as it's rendered in
+the wiki, and if you could even resolve the conflict using the web
+interface. Not 100% sure as I've not gotten into this situation yet.
+
+--[[Joey]]