rewrote number parsing: convert only completely parsed numbers

* We are now buffering until the end of the number constant before converting
  it to an integer or float.
* This way, the integer part of a floating point number does not necessarily
  have to fit into an teco_int_t -- floats can approximate very large numbers.
  When detecting a floating point number, we're parsing into floats from the
  very beginning.
  The behavior has been compared with Lua 5.4 and a test case has been added.
* Also, integers constants are now checked for overflows.
  Otherwise, constants could be arbitrarily long and the result would
  be unpredictable.
  Operations on integers are still not checked for overflows, though, since
  you might want to rely on it, e.g. for overlapping integers. But that
  might actually be pretty useless, especially with signed integers, so we
  might want to change that in the future.
* Having both overflow checks and increased floating point constant range
  is the reason we cannot practically immediately parse into a number,
  not even if we kept both an integer and float accumulator at the same time.
* Also fixes a memory corruption issue that existed since
  4aa2b717f9e050783b567540d9bb0193d692e3e9.
  You must emit undo tokens for fields in state machines only if
  ctx->parent.must_undo, i.e. if the state machine is not temporary as
  during a macro call.
  A test case has been added.

1 files changed, 4 insertions, 6 deletions

diff --git a/src/parser.h b/src/parser.h
index c134d94..8303a2d 100644
--- a/src/parser.h
+++ b/src/parser.h
@@ -441,14 +441,12 @@ typedef struct {
 	 * In case of 0, "^R" determines the radix (10 by default).
 	 */
 	guint radix;
-	/** accumulator for parsing numbers */
-	teco_int_t value;
 	/**
-	 * if not null, value/divisor is the floating point number
-	 * FIXME: Perhaps roll this into a bitfield with radix.
-	 * The entire teco_machine_number_t could be undone.
+	 * Accumulator for integer constants.
+	 * It is not parsed immediately, so we can handle integer
+	 * and float constants differently.
 	 */
-	guint divisor;
+	teco_string_t value;
 } teco_machine_number_t;
 
 /**