Module Surface.Lexer_common

Auxiliary functions used by all lexers.

type lexing_context =
  1. | Law
  2. | Raw
  3. | Code
  4. | Directive
  5. | Directive_args
  6. | Inactive
val with_lexing_context : Catala_utils.File.t -> (unit -> 'a) -> 'a

Initialises the lexing context during the call of the supplied function, which is required for using the lexer. Calls can be nested. Upon termination, emits a warning if the lexer is not in a consistent state (Law context, no pending code content)

val context : lexing_context Stdlib.ref

Reference, used by the lexer as the mutable state to distinguish whether it is lexing code or law.

val context_start_pos : (Stdlib.Lexing.position * Stdlib.Lexing.position) Stdlib.ref

The position of the opening token of the last opened context

val update_acc : Sedlexing.lexbuf -> unit

Updates the current code buffer with the current lexeme. The underlying buffer is used to accumulate the string representation of the body of code being lexed. This string representation is used in the literate programming backends to faithfully capture the spacing pattern of the original program

val flush_acc : unit -> string

Flushes the code buffer and returns its contents (see update_acc)

exception Lexing_error of Catala_utils.Pos.t * string
val raise_lexer_error : Catala_utils.Pos.t -> string -> 'a

Error-generating helper

val token_list_language_agnostic : (string * Tokens.token) list

Associative list matching each punctuation string part of the Catala syntax with its Surface.Parser token. Same for all the input languages (English, French, etc.)

val calc_precedence : string -> int

Calculates the precedence according a matched regex of the form : '#+'

val get_law_heading : Sedlexing.lexbuf -> Tokens.token

Gets the LAW_HEADING token from the current lexbuf

type line_token =
  1. | LINE_INLINE_TEST
    (*

    ```catala-test-cli

    *)
  2. | LINE_BLOCK_END
    (*

    ```

    *)
  3. | LINE_INCLUDE of string
    (*

    > Include foo.catala_en

    *)
  4. | LINE_MODULE_DEF of string * bool
    (*

    > Module Xxx external

    *)
  5. | LINE_MODULE_USE of string
    (*

    > Using Xxx as Yyy

    *)
  6. | LINE_TEST_ATTRIBUTE
    (*

    any line containing a #test attribute

    *)
  7. | LINE_ANY
    (*

    anything else

    *)

Simplified tokens for dependency extraction

module type LocalisedLexer = sig ... end