survex issues on Chinese Windows

[Olly Betts]

On Thu, Aug 18, 2016 at 01:44:50AM +0100, Wookey wrote:
> On any version of survex on any OS: If you process a file which has a
> *begin string which contains UTF-8 chinese characters, e.g.:
> *begin 洞
> 
> you get:
> ChineseTest.svx:1:8: error: Character “�” not allowed in station name (use *SET NAMES to set allowed characters)
> 
> Which I guess is technically correct, but very unhelpful to chinese
> people. Do we do the same ting to anypone who used accented characters
> in latin languages, or cyrillic characters?

Yes, it's only ASCII alphanumerics (and nobody's mentioned it as an
issue previously).

> Is it reasonable to restrict survey names to US-ASCII characters in
> these days of UTF-8 in a nominally internationalised program?  Perhaps
> more chars should be added to the list of chars allowed in survey and
> station names?

Not being able to use accented forms is probably a fairly minor
annoyance (as there are standard ways to write most Latin alphabet
languages with just ASCII), but not being able to use any of your
alphabet is more of a problem.

One issue is that currently there's no character encoding info in Survex
files, so we don't know what those bytes represent at all.  The lack of
encoding information is already an issue (e.g. for displaying the title
in Aven), so we probably should add a new "*encoding" command.

Once we know the encoding we could do a full-Unicode "is alphanumeric"
(or just treat non-ASCII values as valid in names perhaps).

> On Windows 7: 32bit machine, with some kind of chinese locale set - I
> don't know how to get the definitive info on this.
> 
> With survex 1.2.24:
[...]
> Upgrading to survex 1.2.27:
> Installer works in Chinese.
> * Opening from a Chinese path is now OK.
> 
> * Aven menu names and items and errors are all in English.

OK, there's a bug in mapping language codes to strings.  For now you can
set SURVEXLANG=zh_CN in the environment to work around this (details of
how to do that are in the manual).

> * Double-click still opens in notepad. I.e existing 'process' is not
> overwritten with new 'run aven-on-doubleclick'. Perhpas this is
> intentional?

It's meant to update existing actions, but perhaps that doesn't actually
work for some reason.  Uninstalling and reinstalling will probably force
an update.

> On Windows 8, in US-english, 64-bit machine:
> 
> Log output displayed in aven is selectable but not ctrl-C copyable or
> right-click 'copy' copyable.  

That's not implemented anywhere, FWIW, but probably ought to be
supported.

If you want to just save the whole log, "Save Log" is simpler though,
and already works.

Cheers,
    Olly