Skip to content
Snippets Groups Projects
  1. Sep 22, 2022
    • Sven M. Hallberg's avatar
      fail packrat parsers if they need more input · 866fd4d6
      Sven M. Hallberg authored
      This commit changes the contract for the combinator parse functions:
      
       (1) The input state on failure must retain valid overrun and last_chunk
           fields. The latter is never changed, but overrun would be cleared by
           various combinators that backtrack in case of failure. All other
           fields of the input stream are still considered indeterminate after a
           failed parse.
      
       (2) If an overrun condition is encountered before the final chunk
           (last_chunk is false), the parse *must* fail. A helper want_suspend()
           is introduced as a shorthand for this check.
      
      Fixes the packrat/iterative/dummy test.
      866fd4d6
  2. Jun 04, 2021
  3. May 08, 2021
    • picomeg's avatar
      Parsers use backend-vtable pointers without needing the enum. TODO: · 5c30b9c7
      picomeg authored
      right now there is duplication as the enum backend value is still
      present- need to find out if it's OK to make a breaking change for
      anyone who is for reasons known only to themselves chekcing which
      backend enum value is one a parser they may have created and compiled...
      5c30b9c7
  4. Nov 23, 2019
    • xentrac's avatar
      Fix #19, GLR backend reaches unreachable code · 9e662b68
      xentrac authored
      Original behavior:
      
          hammer/build/debug/src/bindings/python$ LD_LIBRARY_PATH=. gdb python
          ...
          (gdb) r
          ...
          Python 2.7.6 (default, Nov 12 2018, 20:00:40)
          [GCC 4.8.4] on linux2
          Type "help", "copyright", "credits" or "license" for more information.
          >>> if 1:
          ...     import hammer as h
          ...     h.choice(h.ch_range('0', '9'), h.ch_range('A', 'Z'), h.ch_range('a', 'z')).compile(h._PB_GLR)
          ...
      
          Program received signal SIGSEGV, Segmentation fault.
          0xb79ecab2 in collect_nts (grammar=0x836abe0, symbol=0xb7d550a4)
              at build/debug/src/cfgrammar.c:120
          120	      for(x = (*s)->items; *x != NULL; x++) {
          (gdb) bt
          #0  0xb79ecab2 in collect_nts (grammar=0x836abe0, symbol=0xb7d550a4)
              at build/debug/src/cfgrammar.c:120
          #1  0xb79ecacd in collect_nts (grammar=0x836abe0, symbol=0x836ab58)
              at build/debug/src/cfgrammar.c:121
          #2  0xb79ec90a in h_cfgrammar_ (mm__=0xb79ff4b4 <system_allocator>,
              desugared=0x836ab58) at build/debug/src/cfgrammar.c:66
          #3  0xb79e8207 in h_lalr_compile (mm__=0xb79ff4b4 <system_allocator>,
              parser=0x836ab40, params=0x0) at build/debug/src/backends/lalr.c:280
          #4  0xb79e634a in h_glr_compile (mm__=0xb79ff4b4 <system_allocator>,
              parser=0x836ab40, params=0x0) at build/debug/src/backends/glr.c:15
          #5  0xb79f0eef in h_compile__m (mm__=0xb79ff4b4 <system_allocator>,
              parser=0x836ab40, backend=PB_GLR, params=0x0)
              at build/debug/src/hammer.c:97
          #6  0xb79f0e9d in h_compile (parser=parser@entry=0x836ab40, backend=PB_GLR,
              params=params@entry=0x0) at build/debug/src/hammer.c:92
          #7  0xb7a54ca4 in HParser__compile (backend=<optimized out>, self=0x836ab40)
              at hammer_wrap.c:3567
      
      New behavior:
      
          hammer/build/debug/src/bindings/python$ LD_LIBRARY_PATH=. gdb python
          ...
          (gdb) r
          ...
          Python 2.7.6 (default, Nov 12 2018, 20:00:40)
          [GCC 4.8.4] on linux2
          Type "help", "copyright", "credits" or "license" for more information.
          >>> if 1:
          ...     import hammer as h
          ...     h.choice(h.ch_range('0', '9'), h.ch_range('A', 'Z'), h.ch_range('a', 'z')).compile(h._PB_GLR)
          ...
          True
          >>> ^D
          [Inferior 1 (process 19621) exited normally]
          (gdb) quit
      
      After thrashing about for a few hours, this was the crucial clue:
      
          >>> import hammer as h
          >>> h.choice(h.ch('0')).compile(h._PB_GLR)
          ==18856== Conditional jump or move depends on uninitialised value(s)
          ==18856==    at 0x4A34FE9: h_desugar (desugar.c:7)
          ==18856==    by 0x4A2D150: h_desugar_augmented (lalr.c:261)
          ==18856==    by 0x4A2D1F7: h_lalr_compile (lalr.c:280)
          ==18856==    by 0x4A2B349: h_glr_compile (glr.c:15)
          ==18856==    by 0x4A35EEE: h_compile__m (hammer.c:97)
          ==18856==    by 0x4A35E9C: h_compile (hammer.c:92)
          ==18856==    by 0x49B0CA3: HParser__compile (hammer_wrap.c:3567)
      
      The particular thing that it's saying is uninitialized seems to be
      
            if(parser->desugared == NULL) {
      
      The `parser` in question is the `HParser` we're trying to desugar,
      which is presumably the choice object created by `h.choice`, which
      seems to be invoking `h_choice__a`:
      
          def choice(*args): return _h_choice__a(list(args))
      
      That was implemented as follows:
      
          HParser* h_choice__a(void *args[]) {
            return h_choice__ma(&system_allocator, args);
          }
      
          HParser* h_choice__ma(HAllocator* mm__, void *args[]) {
            size_t len = -1; // because do...while
            const HParser *arg;
      
            do {
              arg=((HParser **)args)[++len];
            } while(arg);
      
            HSequence *s = h_new(HSequence, 1);
            s->p_array = h_new(HParser *, len);
      
            for (size_t i = 0; i < len; i++) {
              s->p_array[i] = ((HParser **)args)[i];
            }
      
            s->len = len;
            HParser *ret = h_new(HParser, 1);
            ret->vtable = &choice_vt;
            ret->env = (void*)s;
            ret->backend = PB_MIN;
            return ret;
          }
      
      Indeed it does not seem to have been initializing `desugared`.  Fixing
      this cures this symptom.
      
      Other things it's probably worth checking out:
      
      - Are there other places where we create HParser objects where one or
        more fields may be uninitialized?
      - Perhaps `h_new` should zero the memory it returns, since it's only
        used for fixed-size objects and not things like variable-size
        character buffers?
      9e662b68
  5. Dec 20, 2015
  6. Oct 03, 2015
  7. Nov 20, 2013
  8. May 25, 2013
  9. May 24, 2013
  10. May 20, 2013
  11. May 10, 2013
  12. Apr 27, 2013
  13. Apr 23, 2013
  14. Mar 17, 2013
  15. Feb 03, 2013
  16. Dec 18, 2012
  17. Oct 10, 2012
  18. May 26, 2012
Loading