I committed the fix for bug #19 without a test because I didn't know
how our test worked yet; here's a test.

A somewhat more desirable way to do this would be to commit the
test *first*, marked as "incomplete" with `g_test_incomplete()` (an
expected failure).  However, `g_test_incomplete()` does not handle
segfaults!  There's no way to mark a segfaulting test as an "expected
segfault".  So if you want to verify that this test reveals the bug,
you'll need to `git show thiscommit | patch -p1` or something, in a
tree that doesn't have the fix applied.  Or you can comment out the
fix, I guess.

Original behavior:

    hammer/build/debug/src/bindings/python$ LD_LIBRARY_PATH=. gdb python
    ...
    (gdb) r
    ...
    Python 2.7.6 (default, Nov 12 2018, 20:00:40)
    [GCC 4.8.4] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> if 1:
    ...     import hammer as h
    ...     h.choice(h.ch_range('0', '9'), h.ch_range('A', 'Z'), h.ch_range('a', 'z')).compile(h._PB_GLR)
    ...

    Program received signal SIGSEGV, Segmentation fault.
    0xb79ecab2 in collect_nts (grammar=0x836abe0, symbol=0xb7d550a4)
        at build/debug/src/cfgrammar.c:120
    120	      for(x = (*s)->items; *x != NULL; x++) {
    (gdb) bt
    #0  0xb79ecab2 in collect_nts (grammar=0x836abe0, symbol=0xb7d550a4)
        at build/debug/src/cfgrammar.c:120
    #1  0xb79ecacd in collect_nts (grammar=0x836abe0, symbol=0x836ab58)
        at build/debug/src/cfgrammar.c:121
    #2  0xb79ec90a in h_cfgrammar_ (mm__=0xb79ff4b4 <system_allocator>,
        desugared=0x836ab58) at build/debug/src/cfgrammar.c:66
    #3  0xb79e8207 in h_lalr_compile (mm__=0xb79ff4b4 <system_allocator>,
        parser=0x836ab40, params=0x0) at build/debug/src/backends/lalr.c:280
    #4  0xb79e634a in h_glr_compile (mm__=0xb79ff4b4 <system_allocator>,
        parser=0x836ab40, params=0x0) at build/debug/src/backends/glr.c:15
    #5  0xb79f0eef in h_compile__m (mm__=0xb79ff4b4 <system_allocator>,
        parser=0x836ab40, backend=PB_GLR, params=0x0)
        at build/debug/src/hammer.c:97
    #6  0xb79f0e9d in h_compile (parser=parser@entry=0x836ab40, backend=PB_GLR,
        params=params@entry=0x0) at build/debug/src/hammer.c:92
    #7  0xb7a54ca4 in HParser__compile (backend=<optimized out>, self=0x836ab40)
        at hammer_wrap.c:3567

New behavior:

    hammer/build/debug/src/bindings/python$ LD_LIBRARY_PATH=. gdb python
    ...
    (gdb) r
    ...
    Python 2.7.6 (default, Nov 12 2018, 20:00:40)
    [GCC 4.8.4] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> if 1:
    ...     import hammer as h
    ...     h.choice(h.ch_range('0', '9'), h.ch_range('A', 'Z'), h.ch_range('a', 'z')).compile(h._PB_GLR)
    ...
    True
    >>> ^D
    [Inferior 1 (process 19621) exited normally]
    (gdb) quit

After thrashing about for a few hours, this was the crucial clue:

    >>> import hammer as h
    >>> h.choice(h.ch('0')).compile(h._PB_GLR)
    ==18856== Conditional jump or move depends on uninitialised value(s)
    ==18856==    at 0x4A34FE9: h_desugar (desugar.c:7)
    ==18856==    by 0x4A2D150: h_desugar_augmented (lalr.c:261)
    ==18856==    by 0x4A2D1F7: h_lalr_compile (lalr.c:280)
    ==18856==    by 0x4A2B349: h_glr_compile (glr.c:15)
    ==18856==    by 0x4A35EEE: h_compile__m (hammer.c:97)
    ==18856==    by 0x4A35E9C: h_compile (hammer.c:92)
    ==18856==    by 0x49B0CA3: HParser__compile (hammer_wrap.c:3567)

The particular thing that it's saying is uninitialized seems to be

      if(parser->desugared == NULL) {

The `parser` in question is the `HParser` we're trying to desugar,
which is presumably the choice object created by `h.choice`, which
seems to be invoking `h_choice__a`:

    def choice(*args): return _h_choice__a(list(args))

That was implemented as follows:

    HParser* h_choice__a(void *args[]) {
      return h_choice__ma(&system_allocator, args);
    }

    HParser* h_choice__ma(HAllocator* mm__, void *args[]) {
      size_t len = -1; // because do...while
      const HParser *arg;

      do {
        arg=((HParser **)args)[++len];
      } while(arg);

      HSequence *s = h_new(HSequence, 1);
      s->p_array = h_new(HParser *, len);

      for (size_t i = 0; i < len; i++) {
        s->p_array[i] = ((HParser **)args)[i];
      }

      s->len = len;
      HParser *ret = h_new(HParser, 1);
      ret->vtable = &choice_vt;
      ret->env = (void*)s;
      ret->backend = PB_MIN;
      return ret;
    }

Indeed it does not seem to have been initializing `desugared`.  Fixing
this cures this symptom.

Other things it's probably worth checking out:

- Are there other places where we create HParser objects where one or
  more fields may be uninitialized?
- Perhaps `h_new` should zero the memory it returns, since it's only
  used for fixed-size objects and not things like variable-size
  character buffers?

In Python 2.x an unprefixed string literal produces a byte string.
In Python 3.x an unprefixed string literal produces a textual string.

To produce a byte string in both a b prefix is needed, e.g. b'foo'.
Since I believe Hammer works predominantly with byte strings I have used
b prefixes throughout.

This removes any doubts about what type of string is in use.

This allows the library to be built and tested with a non-default
version of CPython, e.g.

scons bindings=python python=python3.6
scons bindings=python python=python3.6 testpython

In Python 3.x

- int and long types are unified. The unified type is called int.
- the text string type (unicode) is renamed to str.
- the byte string type (str) is renamed to bytes.
- chr returns a text string (i.e. str)
- xrange is renamed to range.
- dict.has_key() is removed
-

These have no effect in Python 3.x, they are the default. Enabling them
in Python 2.x, enabling them in Python 2.x allows single source
compatiblity.

I did this in a hurry, but it appears to work locally. I tested it with
a couple boring custom token type printers.

To homogenize the file and allow keys/strings to be searched easily.

We disable:
- the tests (which require glib) although they can be
  reactivated with the `--tests` command line flag
- shared library (lack of export symbol declarations
  means that although it can be built, no symbol is
  exported and therefore it can't be used)

The `install` target installs the library and headers
under the `build` folder, because it's a traditional practice
to move libraries to a central location on Windows, unless
you are using cygwin. In which case pass `prefix` to the
command line.

We adapt tools\windows\build_examples.bat to take the library
that is built using scons or using tools\windows\build.bat

We port registry by importing the (public domain) openbsd implementation
of the tfind/tsearch POSIX binary tree search functions.

These are only necessary when building on non-posix platforms

MSVC was complaining that the `tail` variable was potentially
uninitialized in the while branch. Since the while loop is actually
coupled to the if (head != NULL) that initializes the tail variable,
we move them together, which makes the warning disappear.

As MSVC doesn't implement C99, variable-length arrays
are not supported. We use _alloca instead.