documented the automatic EOL translation feature

3 files changed, 75 insertions, 7 deletions

diff --git a/doc/sciteco.7.template b/doc/sciteco.7.template
index a342f5d..288c3dd 100644
--- a/doc/sciteco.7.template
+++ b/doc/sciteco.7.template
@@ -111,9 +111,7 @@ The most prominent is the Escape key \(em it is translated to
 code 27.
 The backspace key will always be translated to code 8, and the Tab key
 to code 9.
-Last but not least, the Return key is translated to the current
-buffer's end of line sequence (linefeed, carriage return followed
-by linefeed or just carriage return).
+The return/enter key is always translated to code 10 (line feed).
 Naturally, all of these control codes can also be typed using a
 Control-key combination (e.g. CTRL+I for the tab character) and
 there is often an equivalent typed with the caret character
@@ -278,6 +276,22 @@ effectively switches between undo and redo modes.
 The modifier also influences the behaviour of the TAB key.
 T}
 T{
+Insert end of line sequence
+T};10;^J, Return, Enter;T{
+Everywhere
+.br
+(no automatic EOL translation)
+T};T{
+If automatic end of line sequence translation is disabled,
+the return key will insert the current buffer's end of line
+sequence (line feed; carriage return, line feed; or
+carriage return) according to the
+.B SCI_GETEOLMODE
+Scintilla message.
+In auto-EOL-translation mode, the return key will always
+insert a line feed.
+T}
+T{
 Rub out character
 T};8;^H, Backspace;T{
 Everywhere
@@ -674,9 +688,46 @@ bytes depending on the document's encoding.
 Consequently when querying the code at a character position
 or inserting characters by code, the code may be an Unicode
 codepoint instead of byte-sized integer.
-\# FIXME: this is currently broken/untested
 Currently however, \*(ST will only handle ASCII files.
 .LP
+To simplify working with files using different end of line
+sequences (combinations of carriage return and line feed),
+\*(ST implements an \fIautomatic EOL translation\fP
+feature.
+When loading files with the \fBEB\fP and \fBEQ\fP commands \(em
+or when reading from an external process with the \fBEC\fP and
+\fBEG\fP commands \(em
+\*(ST will normalize all end of line sequences to line feeds.
+It will also detect, correct and warn about inconsistent EOL sequences
+in the source file or process output.
+EOL normalization is thus a lossy process.
+The file's original EOL mode as guessed by \*(ST is still saved
+as the corresponding Scintilla document's EOL mode
+(see \fBSCI_SETEOLMODE\fP and the \fBEL\fP command).
+If the EOL mode could not be guessed, the document will retain
+a platform-default EOL mode (e.g. carriage return, line feed on
+Windows and line feed on UNIX).
+The EOL mode is used when saving files with the \fBEW\fP and \fBE%\fP
+commands or when writing to an external process with the \fBEC\fP and \fBEG\fP
+commands in order to reconstruct the original EOL characters.
+Translation on write-out will also work when the source document
+has inconsistent or mixed line breaks.
+Therefore macros can assume each line to be terminated by
+a single line feed character while working with files that have
+different EOL styles.
+The feature is enabled by default but is controlled by bit 4
+of the \fBED\fP flag setting.
+It can be turned off by executing:
+.EX
+16,0ED
+.EE
+This may be beneficial when working with very large files \(em
+as the EOL translation imposes a performance penalty \(em
+or non-text (binary) files.
+No automatic guessing of documents' EOL mode is performed
+when disabling this feature, so all documents will have
+platform-default EOL modes set.
+.LP
 For more details, please refer to this manual's command
 reference.
 .
diff --git a/src/parser.cpp b/src/parser.cpp
index 7d242fd..6ca6652 100644
--- a/src/parser.cpp
+++ b/src/parser.cpp
@@ -1852,7 +1852,7 @@ StateECommand::custom(gchar chr)
 	 * [off,]on ED
 	 * ED -> flags
 	 *
-	 * With arguments, the command will set the ED flags.
+	 * With arguments, the command will set the \fBED\fP flags.
 	 * <flags> is a bitmap of flags to set.
 	 * Specifying one argument to set the flags is a special
 	 * case of specifying two arguments that allow to control
@@ -1866,6 +1866,8 @@ StateECommand::custom(gchar chr)
 	 * Without any argument ED returns the current flags.
 	 *
 	 * Currently, the following flags are used by \*(ST:
+	 *   - 16: Enable/disable automatic translation of end of
+	 *     line sequences to and from line feed.
 	 *   - 32: Enable/Disable buffer editing hooks
 	 *     (via execution of macro in global Q-Register \(lqED\(rq)
 	 *   - 64: Enable/Disable function key macros
@@ -1875,6 +1877,9 @@ StateECommand::custom(gchar chr)
 	 *
 	 * The features controlled thus are discribed in other sections
 	 * of this manual.
+	 *
+	 * The default value of the \fBED\fP flags is 16
+	 * (only automatic EOL translation enabled).
 	 */
 	case 'D':
 		BEGIN_EXEC(&States::start);
@@ -2006,8 +2011,8 @@ StateECommand::custom(gchar chr)
 	 * Like the Scintilla message, <EL> does \fBnot\fP change the
 	 * characters in the current document.
 	 * If automatic EOL translation is activated (which is the default),
-	 * \*(ST will however use this information when saving and writing
-	 * out files.
+	 * \*(ST will however use this information when saving files or
+	 * writing to external processes.
 	 *
 	 * With one argument, the EOL mode is set according to these
 	 * constants:
diff --git a/src/spawn.cpp b/src/spawn.cpp
index af42a95..c1ffce8 100644
--- a/src/spawn.cpp
+++ b/src/spawn.cpp
@@ -133,8 +133,17 @@ parse_shell_command_line(const gchar *cmdline, GError **error)
  * and its standard input stream is fed with data from the
  * current document, i.e. text is piped into the external
  * program.
+ * When automatic EOL translation is enabled, this will
+ * translate all end of line sequences according to the
+ * source document's EOL mode (see \fBEL\fP command).
+ * For instance when piping from a document with DOS
+ * line breaks, the receiving program will only be sent
+ * DOS line breaks.
  * The process' standard output stream is also redirected
  * and inserted into the current document.
+ * End of line sequences are normalized accordingly
+ * but the EOL mode guessed from the program's output is
+ * \fBnot\fP set on the document.
  * The process' standard error stream is discarded.
  * If data is piped into the external program, its output
  * replaces that data in the buffer.
@@ -455,6 +464,9 @@ gerror:
  * the current buffer position and writing process output
  * to the Q-Register <q>.
  * In other words, the current buffer is not modified by EG.
+ * Also since EG replaces the string value of <q>, the register's
+ * EOL mode is set to the mode guessed from the external program's
+ * output.
  *
  * The register <q> is defined if it does not already exist.
  */