indicate how some goods can be taken out of the po plugin without using po files

[git.ikiwiki.info.git] / doc / bugs / UTF-8_BOM_showing_up_inside_a_page__63__.mdwn

diff --git a/doc/bugs/UTF-8_BOM_showing_up_inside_a_page__63__.mdwn b/doc/bugs/UTF-8_BOM_showing_up_inside_a_page__63__.mdwn
index 5324f799187bb828feab0d077cc5c22affff8fe4..3b9c9662d6e6db411b6d1a468caac53c4377ed7b 100644 (file)
--- a/doc/bugs/UTF-8_BOM_showing_up_inside_a_page__63__.mdwn
+++ b/doc/bugs/UTF-8_BOM_showing_up_inside_a_page__63__.mdwn
@@ -1,4 +1,4 @@
-I have a git-backed ikiwiki install, and when I commit and push a file from an x86 host (LANG=en_US.UTF-8) to the Ikiwiki box, which is Debian GNU/Linux on Sparc, I sometimes get unusual characters (ef bb ff) before the first character of the wiki text.  It seems that this is a UTF-8 "byte order mark" that is getting inserted automatically into the .wiki file by my editor: http://vim.wikia.com/wiki/VimTip246#Tip:_.23246_-_Working_with_Unicode 
+I have a git-backed ikiwiki install, and when I commit and push a file from an x86 host (LANG=en_US.UTF-8) to the Ikiwiki box, which is Debian GNU/Linux on Sparc, I sometimes get unusual characters (ef bb ff) before the first character of the wiki text.  It seems that this is a UTF-8 "byte order mark" that is getting inserted automatically into the .wiki file by my editor: http://vim.wikia.com/wiki/VimTip246#Tip:_.23246_-_Working_with_Unicode 
 
 Example:
 
@@ -11,3 +11,28 @@ deal with it, or should I make sure to strip it out before committing?
 > plugin could be written to s/// them out as a filter.
 >
 > I'm unsure if ikiwiki should do this by default. --[[Joey]]
+
+> Looked at this some more. It seems this would be a browser bug, after
+> all, it's not displaying the BOM properly.
+> To test, I've added a BOM to this file. 
+> 
+> Well, this page looks ok in epiphany and w3m, even with the BOM. Epiphany
+> incorrectly displays it as a space (not zero-width). In w3m in a unicode
+> xterm, it's invisible. What's going on is that <FEFF> is only a BOM at
+> the very beginning of the file. Otherwise, it should be treated as a
+> zero-width, non-breaking space. Ie, invisible. Any browsers that display
+> it otherwise seem to be broken. 
+> 
+> I'm having a hard time with the idea that any program that reads utf-8
+> data from a file and sticks it in the middle on another, output, utf-8
+> file, is broken if it doesn't strip the BOM. It could be argued that
+> programs should do that; it could be argued that perl should strip the
+> BOM from the beginning of a file whenever reading a file in utf8 mode, to
+> avoid all perl programs needing to do this on their own. Or it could be
+> argued that requiring all programs do this is silly, and that the BOM was
+> designed so you didn't need to strip it.
+> 
+> After consideration, I prefer this last argument, so I prefer not to
+> make ikiwiki stip utf8 BOMS. Calling this bug [[done]].
+> 
+> --[[Joey]]