What tokens are in the end of line?What are the exact semantics of detokenize?Test whether end will really end the runWhy are some characters not allowed in command sequencesHow TeX reads what you typeWhy does the TeX scanner process tokens for register numbers and macro names differently?How to pass a control-sequence as an argument to a macro that expects whitespace after the argument?What state changes are caused by input?A line starting with a spaceHow is the TeX engine informed of the OS's end-of-line marker and file encoding?About what TeX does after it finds commented conditionals with ltxdoc
Why colon to denote that a value belongs to a type?
How many chess players are over 2500 Elo?
Dictionary size reduces upon increasing one element
Array Stutter Implementation
Is there a public standard for 8 and 10 character grid locators?
Where did Wilson state that the US would have to force access to markets with violence?
How long does it take to crack RSA 1024 with a PC?
Why is this Simple Puzzle impossible to solve?
Placing bypass capacitors after VCC reaches the IC
Would the Geas spell work in a dead magic zone once you enter it?
Is there a general effective method to solve Smullyan style Knights and Knaves problems? Is the truth table method the most appropriate one?
How does an ARM MCU run faster than the external crystal?
How do I subvert the tropes of a train heist?
Is there an efficient way to replace text matching the entire content of one file with the entire content of another file?
Looking for a soft substance that doesn't dissolve underwater
What is the object moving across the ceiling in this stock footage?
How bitcoin nodes update UTXO set when their latests blocks are replaced?
General purpose replacement for enum with FlagsAttribute
Employer demanding to see degree after poor code review
Tic-tac-toe for the terminal, written in C
What are the benefits of cryosleep?
What is the largest (size) solid object ever dropped from an airplane to impact the ground in freefall?
Integrating an absolute function using Mathematica
Is it ok to put a subplot to a story that is never meant to contribute to the development of the main plot?
What tokens are in the end of line?
What are the exact semantics of detokenize?Test whether end will really end the runWhy are some characters not allowed in command sequencesHow TeX reads what you typeWhy does the TeX scanner process tokens for register numbers and macro names differently?How to pass a control-sequence as an argument to a macro that expects whitespace after the argument?What state changes are caused by input?A line starting with a spaceHow is the TeX engine informed of the OS's end-of-line marker and file encoding?About what TeX does after it finds commented conditionals with ltxdoc
Consider the following test.tex
:
ERROR
a
If we run tex
on it, and start skipping tokens one by one, we get this:
(test.tex
! Undefined control sequence.
l.1 ERROR
? 1
l.2 a
? 1
l.2 a
? 1
)
As we see, the first 1
command skips end-of-line.
Then we see that TeX has just read a
and is about to read end-of-line character. We skip it with the second 1
command.
Then we see the same picture. As if nothing happened. And only after the third 1
command we see that TeX has passed further.
Why TeX skips end-of-line after ERROR
from the first time, but after a
only from the second time?
Also, it is strange that if after the final *
, which is not shown in the output, we say end
, tex
says:
*end
<*> end
?
instead of terminating.
EDIT
Consider the following 6 test files:
control-word_at_eol.tex
control-word_not_at_eol.tex
non-space_at_eol.tex
non-space_not_at_eol.tex
space_at_eol.tex
space_not_at_eol.tex
Here are results that I obtained:
control-word_at_eol.tex
: space and ^^M is skipped right after reading control word
control-word_not_at_eol.tex
: space is not skipped right after reading control word
non-space_at_eol.tex
: space is skipped right after reading control symbol, ^^M not skipped
non-space_not_at_eol.tex
: space is not skipped right after reading control symbol
space_at_eol.tex
: space, ^^M and (!!!) name of control symbol are skipped right after reading control symbol
space_not_at_eol.tex
: space is not skipped right after reading control symbol
tex-core
add a comment |
Consider the following test.tex
:
ERROR
a
If we run tex
on it, and start skipping tokens one by one, we get this:
(test.tex
! Undefined control sequence.
l.1 ERROR
? 1
l.2 a
? 1
l.2 a
? 1
)
As we see, the first 1
command skips end-of-line.
Then we see that TeX has just read a
and is about to read end-of-line character. We skip it with the second 1
command.
Then we see the same picture. As if nothing happened. And only after the third 1
command we see that TeX has passed further.
Why TeX skips end-of-line after ERROR
from the first time, but after a
only from the second time?
Also, it is strange that if after the final *
, which is not shown in the output, we say end
, tex
says:
*end
<*> end
?
instead of terminating.
EDIT
Consider the following 6 test files:
control-word_at_eol.tex
control-word_not_at_eol.tex
non-space_at_eol.tex
non-space_not_at_eol.tex
space_at_eol.tex
space_not_at_eol.tex
Here are results that I obtained:
control-word_at_eol.tex
: space and ^^M is skipped right after reading control word
control-word_not_at_eol.tex
: space is not skipped right after reading control word
non-space_at_eol.tex
: space is skipped right after reading control symbol, ^^M not skipped
non-space_not_at_eol.tex
: space is not skipped right after reading control symbol
space_at_eol.tex
: space, ^^M and (!!!) name of control symbol are skipped right after reading control symbol
space_not_at_eol.tex
: space is not skipped right after reading control symbol
tex-core
correction) in file 2) space is skipped, it's ^^M which is not, in file 3) space not skipped it's ^^M. file 6) is same as 2 space skkiped but ^^M not.
– touhami
May 22 at 23:07
add a comment |
Consider the following test.tex
:
ERROR
a
If we run tex
on it, and start skipping tokens one by one, we get this:
(test.tex
! Undefined control sequence.
l.1 ERROR
? 1
l.2 a
? 1
l.2 a
? 1
)
As we see, the first 1
command skips end-of-line.
Then we see that TeX has just read a
and is about to read end-of-line character. We skip it with the second 1
command.
Then we see the same picture. As if nothing happened. And only after the third 1
command we see that TeX has passed further.
Why TeX skips end-of-line after ERROR
from the first time, but after a
only from the second time?
Also, it is strange that if after the final *
, which is not shown in the output, we say end
, tex
says:
*end
<*> end
?
instead of terminating.
EDIT
Consider the following 6 test files:
control-word_at_eol.tex
control-word_not_at_eol.tex
non-space_at_eol.tex
non-space_not_at_eol.tex
space_at_eol.tex
space_not_at_eol.tex
Here are results that I obtained:
control-word_at_eol.tex
: space and ^^M is skipped right after reading control word
control-word_not_at_eol.tex
: space is not skipped right after reading control word
non-space_at_eol.tex
: space is skipped right after reading control symbol, ^^M not skipped
non-space_not_at_eol.tex
: space is not skipped right after reading control symbol
space_at_eol.tex
: space, ^^M and (!!!) name of control symbol are skipped right after reading control symbol
space_not_at_eol.tex
: space is not skipped right after reading control symbol
tex-core
Consider the following test.tex
:
ERROR
a
If we run tex
on it, and start skipping tokens one by one, we get this:
(test.tex
! Undefined control sequence.
l.1 ERROR
? 1
l.2 a
? 1
l.2 a
? 1
)
As we see, the first 1
command skips end-of-line.
Then we see that TeX has just read a
and is about to read end-of-line character. We skip it with the second 1
command.
Then we see the same picture. As if nothing happened. And only after the third 1
command we see that TeX has passed further.
Why TeX skips end-of-line after ERROR
from the first time, but after a
only from the second time?
Also, it is strange that if after the final *
, which is not shown in the output, we say end
, tex
says:
*end
<*> end
?
instead of terminating.
EDIT
Consider the following 6 test files:
control-word_at_eol.tex
control-word_not_at_eol.tex
non-space_at_eol.tex
non-space_not_at_eol.tex
space_at_eol.tex
space_not_at_eol.tex
Here are results that I obtained:
control-word_at_eol.tex
: space and ^^M is skipped right after reading control word
control-word_not_at_eol.tex
: space is not skipped right after reading control word
non-space_at_eol.tex
: space is skipped right after reading control symbol, ^^M not skipped
non-space_not_at_eol.tex
: space is not skipped right after reading control symbol
space_at_eol.tex
: space, ^^M and (!!!) name of control symbol are skipped right after reading control symbol
space_not_at_eol.tex
: space is not skipped right after reading control symbol
tex-core
tex-core
edited May 22 at 3:32
Igor Liferenko
asked May 21 at 9:15
Igor LiferenkoIgor Liferenko
2,223829
2,223829
correction) in file 2) space is skipped, it's ^^M which is not, in file 3) space not skipped it's ^^M. file 6) is same as 2 space skkiped but ^^M not.
– touhami
May 22 at 23:07
add a comment |
correction) in file 2) space is skipped, it's ^^M which is not, in file 3) space not skipped it's ^^M. file 6) is same as 2 space skkiped but ^^M not.
– touhami
May 22 at 23:07
correction) in file 2) space is skipped, it's ^^M which is not, in file 3) space not skipped it's ^^M. file 6) is same as 2 space skkiped but ^^M not.
– touhami
May 22 at 23:07
correction) in file 2) space is skipped, it's ^^M which is not, in file 3) space not skipped it's ^^M. file 6) is same as 2 space skkiped but ^^M not.
– touhami
May 22 at 23:07
add a comment |
2 Answers
2
active
oldest
votes
You can understand this better with a different input
ERROR xyz
a
If you run tex
on it, you get
This is TeX, Version 3.14159265 (TeX Live 2019) (preloaded format=tex)
(./igor.tex
! Undefined control sequence.
l.1 ERROR
xyz
? 1
l.1 ERROR x
yz
Error lines are divided into two parts (actually three if TeX is in the middle of expansions when it finds an error); here the top line shows how far TeX has gone in the input file, the bottom line shows what's awaiting to be read. Note that ERROR
sits in the top line, so it has already been read and gone. The slight misalignment between the two lines is because TeX represents control words with a trailing space, but the space “is not there” and, anyway, it is no longer relevant because it's in the top line.
With 1
you delete the next token and TeX stops again showing the context as before.
What happens with your attempt?
ERROR
a
Let's see it step-by-step:
! Undefined control sequence.
l.1 ERROR
?
Nothing is shown in the bottom line, because the line has ended. The end-of-line has already been converted to a category code ^^M
and the generated space has been gobbled during tokenization of ERROR
. TeX is in state N (beginning of a new line).
If you type 1
followed by return, you get
? 1
l.2 a
?
TeX has entered state M because it has read a
(and ignored it); the bottom line shows nothing for the same reason as before. Note that the ^^M
is still unread.
Type another 1
to get
? 1
l.2 a
?
The gobbled token is now the space generated by ^^M
and TeX enters state N again.
Type another 1
:
? 1
)
*
The )
means the file has ended; *
means that TeX is awaiting for input. This is again in the format “top line/bottom line”: there was nothing to be ignored, but TeX follows instructions and wants to ignore something.
Type end
:
<*> end
?
Again, “top line/bottom line”: the top line shows what has been read in so far. There is no line number, hence TeX shows <*>
to denote something directly input in the interactive session and what it has ignored following the previous 1
instruction.
Hitting return now shows the *
prompt: TeX is awaiting for input.
Top line does not end with a space in "xyz" example: try double click in terminal at the end of top line - highlighting will begin right afterERROR
, thus there is no space.
– Igor Liferenko
May 22 at 1:17
Ideally, in the original example the^^M
converted to space should have been shown at the end of top line after second1
command (aftera
), because the top line shows how far TeX has gone in the input file. The absence of space can be verified objectively by runninghexdump -Cv test.log
– Igor Liferenko
May 22 at 1:47
Please fix "The top line ends with a space" sentence, as it is wrong. See answer to exercise 8.3 in TeXbook.
– Igor Liferenko
May 22 at 2:58
See EDIT in OP.
– Igor Liferenko
May 22 at 3:32
@IgorLiferenko It was not wrong; I reworded anyway. About the six files, I don't know what to say: I'm not going to download them from an unknown repository
– egreg
May 22 at 8:43
add a comment |
Why TeX skips end-of-line after ERROR from the first time, but after a only from the second time?
TeX removes any spaces from the end of the line (some older implementations erroneously removed tabs as well) and then if endlinechar
is in the 0-255 range that character is added to the end of the line) in plain and latex that defaults to 13 (control M).
As ^M
has catcode 5, it normally acts as space so is gobbled while tokenizing ERROR
and which explains the different behaviour for the first two lines.
if you delete 2 tokens then end
works
! Undefined control sequence.
l.1 ERROR
? 1
l.2 a
? 1
l.2 a
?
)
*end
No pages of output.
using pdftex
here.
with three 1 you get
! Undefined control sequence.
l.1 ERROR
? 1
l.2 a
? 1
l.2 a
? 1
)
*end
<*> end
because the final 1 gobbles the next token but that is the end
that you are about to add....
The difference is because you do not type1
after the last?
.
– Igor Liferenko
May 21 at 9:38
@IgorLiferenko yes I just updated the answer
– David Carlisle
May 21 at 9:39
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "85"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2ftex.stackexchange.com%2fquestions%2f491891%2fwhat-tokens-are-in-the-end-of-line%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
You can understand this better with a different input
ERROR xyz
a
If you run tex
on it, you get
This is TeX, Version 3.14159265 (TeX Live 2019) (preloaded format=tex)
(./igor.tex
! Undefined control sequence.
l.1 ERROR
xyz
? 1
l.1 ERROR x
yz
Error lines are divided into two parts (actually three if TeX is in the middle of expansions when it finds an error); here the top line shows how far TeX has gone in the input file, the bottom line shows what's awaiting to be read. Note that ERROR
sits in the top line, so it has already been read and gone. The slight misalignment between the two lines is because TeX represents control words with a trailing space, but the space “is not there” and, anyway, it is no longer relevant because it's in the top line.
With 1
you delete the next token and TeX stops again showing the context as before.
What happens with your attempt?
ERROR
a
Let's see it step-by-step:
! Undefined control sequence.
l.1 ERROR
?
Nothing is shown in the bottom line, because the line has ended. The end-of-line has already been converted to a category code ^^M
and the generated space has been gobbled during tokenization of ERROR
. TeX is in state N (beginning of a new line).
If you type 1
followed by return, you get
? 1
l.2 a
?
TeX has entered state M because it has read a
(and ignored it); the bottom line shows nothing for the same reason as before. Note that the ^^M
is still unread.
Type another 1
to get
? 1
l.2 a
?
The gobbled token is now the space generated by ^^M
and TeX enters state N again.
Type another 1
:
? 1
)
*
The )
means the file has ended; *
means that TeX is awaiting for input. This is again in the format “top line/bottom line”: there was nothing to be ignored, but TeX follows instructions and wants to ignore something.
Type end
:
<*> end
?
Again, “top line/bottom line”: the top line shows what has been read in so far. There is no line number, hence TeX shows <*>
to denote something directly input in the interactive session and what it has ignored following the previous 1
instruction.
Hitting return now shows the *
prompt: TeX is awaiting for input.
Top line does not end with a space in "xyz" example: try double click in terminal at the end of top line - highlighting will begin right afterERROR
, thus there is no space.
– Igor Liferenko
May 22 at 1:17
Ideally, in the original example the^^M
converted to space should have been shown at the end of top line after second1
command (aftera
), because the top line shows how far TeX has gone in the input file. The absence of space can be verified objectively by runninghexdump -Cv test.log
– Igor Liferenko
May 22 at 1:47
Please fix "The top line ends with a space" sentence, as it is wrong. See answer to exercise 8.3 in TeXbook.
– Igor Liferenko
May 22 at 2:58
See EDIT in OP.
– Igor Liferenko
May 22 at 3:32
@IgorLiferenko It was not wrong; I reworded anyway. About the six files, I don't know what to say: I'm not going to download them from an unknown repository
– egreg
May 22 at 8:43
add a comment |
You can understand this better with a different input
ERROR xyz
a
If you run tex
on it, you get
This is TeX, Version 3.14159265 (TeX Live 2019) (preloaded format=tex)
(./igor.tex
! Undefined control sequence.
l.1 ERROR
xyz
? 1
l.1 ERROR x
yz
Error lines are divided into two parts (actually three if TeX is in the middle of expansions when it finds an error); here the top line shows how far TeX has gone in the input file, the bottom line shows what's awaiting to be read. Note that ERROR
sits in the top line, so it has already been read and gone. The slight misalignment between the two lines is because TeX represents control words with a trailing space, but the space “is not there” and, anyway, it is no longer relevant because it's in the top line.
With 1
you delete the next token and TeX stops again showing the context as before.
What happens with your attempt?
ERROR
a
Let's see it step-by-step:
! Undefined control sequence.
l.1 ERROR
?
Nothing is shown in the bottom line, because the line has ended. The end-of-line has already been converted to a category code ^^M
and the generated space has been gobbled during tokenization of ERROR
. TeX is in state N (beginning of a new line).
If you type 1
followed by return, you get
? 1
l.2 a
?
TeX has entered state M because it has read a
(and ignored it); the bottom line shows nothing for the same reason as before. Note that the ^^M
is still unread.
Type another 1
to get
? 1
l.2 a
?
The gobbled token is now the space generated by ^^M
and TeX enters state N again.
Type another 1
:
? 1
)
*
The )
means the file has ended; *
means that TeX is awaiting for input. This is again in the format “top line/bottom line”: there was nothing to be ignored, but TeX follows instructions and wants to ignore something.
Type end
:
<*> end
?
Again, “top line/bottom line”: the top line shows what has been read in so far. There is no line number, hence TeX shows <*>
to denote something directly input in the interactive session and what it has ignored following the previous 1
instruction.
Hitting return now shows the *
prompt: TeX is awaiting for input.
Top line does not end with a space in "xyz" example: try double click in terminal at the end of top line - highlighting will begin right afterERROR
, thus there is no space.
– Igor Liferenko
May 22 at 1:17
Ideally, in the original example the^^M
converted to space should have been shown at the end of top line after second1
command (aftera
), because the top line shows how far TeX has gone in the input file. The absence of space can be verified objectively by runninghexdump -Cv test.log
– Igor Liferenko
May 22 at 1:47
Please fix "The top line ends with a space" sentence, as it is wrong. See answer to exercise 8.3 in TeXbook.
– Igor Liferenko
May 22 at 2:58
See EDIT in OP.
– Igor Liferenko
May 22 at 3:32
@IgorLiferenko It was not wrong; I reworded anyway. About the six files, I don't know what to say: I'm not going to download them from an unknown repository
– egreg
May 22 at 8:43
add a comment |
You can understand this better with a different input
ERROR xyz
a
If you run tex
on it, you get
This is TeX, Version 3.14159265 (TeX Live 2019) (preloaded format=tex)
(./igor.tex
! Undefined control sequence.
l.1 ERROR
xyz
? 1
l.1 ERROR x
yz
Error lines are divided into two parts (actually three if TeX is in the middle of expansions when it finds an error); here the top line shows how far TeX has gone in the input file, the bottom line shows what's awaiting to be read. Note that ERROR
sits in the top line, so it has already been read and gone. The slight misalignment between the two lines is because TeX represents control words with a trailing space, but the space “is not there” and, anyway, it is no longer relevant because it's in the top line.
With 1
you delete the next token and TeX stops again showing the context as before.
What happens with your attempt?
ERROR
a
Let's see it step-by-step:
! Undefined control sequence.
l.1 ERROR
?
Nothing is shown in the bottom line, because the line has ended. The end-of-line has already been converted to a category code ^^M
and the generated space has been gobbled during tokenization of ERROR
. TeX is in state N (beginning of a new line).
If you type 1
followed by return, you get
? 1
l.2 a
?
TeX has entered state M because it has read a
(and ignored it); the bottom line shows nothing for the same reason as before. Note that the ^^M
is still unread.
Type another 1
to get
? 1
l.2 a
?
The gobbled token is now the space generated by ^^M
and TeX enters state N again.
Type another 1
:
? 1
)
*
The )
means the file has ended; *
means that TeX is awaiting for input. This is again in the format “top line/bottom line”: there was nothing to be ignored, but TeX follows instructions and wants to ignore something.
Type end
:
<*> end
?
Again, “top line/bottom line”: the top line shows what has been read in so far. There is no line number, hence TeX shows <*>
to denote something directly input in the interactive session and what it has ignored following the previous 1
instruction.
Hitting return now shows the *
prompt: TeX is awaiting for input.
You can understand this better with a different input
ERROR xyz
a
If you run tex
on it, you get
This is TeX, Version 3.14159265 (TeX Live 2019) (preloaded format=tex)
(./igor.tex
! Undefined control sequence.
l.1 ERROR
xyz
? 1
l.1 ERROR x
yz
Error lines are divided into two parts (actually three if TeX is in the middle of expansions when it finds an error); here the top line shows how far TeX has gone in the input file, the bottom line shows what's awaiting to be read. Note that ERROR
sits in the top line, so it has already been read and gone. The slight misalignment between the two lines is because TeX represents control words with a trailing space, but the space “is not there” and, anyway, it is no longer relevant because it's in the top line.
With 1
you delete the next token and TeX stops again showing the context as before.
What happens with your attempt?
ERROR
a
Let's see it step-by-step:
! Undefined control sequence.
l.1 ERROR
?
Nothing is shown in the bottom line, because the line has ended. The end-of-line has already been converted to a category code ^^M
and the generated space has been gobbled during tokenization of ERROR
. TeX is in state N (beginning of a new line).
If you type 1
followed by return, you get
? 1
l.2 a
?
TeX has entered state M because it has read a
(and ignored it); the bottom line shows nothing for the same reason as before. Note that the ^^M
is still unread.
Type another 1
to get
? 1
l.2 a
?
The gobbled token is now the space generated by ^^M
and TeX enters state N again.
Type another 1
:
? 1
)
*
The )
means the file has ended; *
means that TeX is awaiting for input. This is again in the format “top line/bottom line”: there was nothing to be ignored, but TeX follows instructions and wants to ignore something.
Type end
:
<*> end
?
Again, “top line/bottom line”: the top line shows what has been read in so far. There is no line number, hence TeX shows <*>
to denote something directly input in the interactive session and what it has ignored following the previous 1
instruction.
Hitting return now shows the *
prompt: TeX is awaiting for input.
edited May 22 at 8:42
answered May 21 at 11:01
egregegreg
744k8919493287
744k8919493287
Top line does not end with a space in "xyz" example: try double click in terminal at the end of top line - highlighting will begin right afterERROR
, thus there is no space.
– Igor Liferenko
May 22 at 1:17
Ideally, in the original example the^^M
converted to space should have been shown at the end of top line after second1
command (aftera
), because the top line shows how far TeX has gone in the input file. The absence of space can be verified objectively by runninghexdump -Cv test.log
– Igor Liferenko
May 22 at 1:47
Please fix "The top line ends with a space" sentence, as it is wrong. See answer to exercise 8.3 in TeXbook.
– Igor Liferenko
May 22 at 2:58
See EDIT in OP.
– Igor Liferenko
May 22 at 3:32
@IgorLiferenko It was not wrong; I reworded anyway. About the six files, I don't know what to say: I'm not going to download them from an unknown repository
– egreg
May 22 at 8:43
add a comment |
Top line does not end with a space in "xyz" example: try double click in terminal at the end of top line - highlighting will begin right afterERROR
, thus there is no space.
– Igor Liferenko
May 22 at 1:17
Ideally, in the original example the^^M
converted to space should have been shown at the end of top line after second1
command (aftera
), because the top line shows how far TeX has gone in the input file. The absence of space can be verified objectively by runninghexdump -Cv test.log
– Igor Liferenko
May 22 at 1:47
Please fix "The top line ends with a space" sentence, as it is wrong. See answer to exercise 8.3 in TeXbook.
– Igor Liferenko
May 22 at 2:58
See EDIT in OP.
– Igor Liferenko
May 22 at 3:32
@IgorLiferenko It was not wrong; I reworded anyway. About the six files, I don't know what to say: I'm not going to download them from an unknown repository
– egreg
May 22 at 8:43
Top line does not end with a space in "xyz" example: try double click in terminal at the end of top line - highlighting will begin right after
ERROR
, thus there is no space.– Igor Liferenko
May 22 at 1:17
Top line does not end with a space in "xyz" example: try double click in terminal at the end of top line - highlighting will begin right after
ERROR
, thus there is no space.– Igor Liferenko
May 22 at 1:17
Ideally, in the original example the
^^M
converted to space should have been shown at the end of top line after second 1
command (after a
), because the top line shows how far TeX has gone in the input file. The absence of space can be verified objectively by running hexdump -Cv test.log
– Igor Liferenko
May 22 at 1:47
Ideally, in the original example the
^^M
converted to space should have been shown at the end of top line after second 1
command (after a
), because the top line shows how far TeX has gone in the input file. The absence of space can be verified objectively by running hexdump -Cv test.log
– Igor Liferenko
May 22 at 1:47
Please fix "The top line ends with a space" sentence, as it is wrong. See answer to exercise 8.3 in TeXbook.
– Igor Liferenko
May 22 at 2:58
Please fix "The top line ends with a space" sentence, as it is wrong. See answer to exercise 8.3 in TeXbook.
– Igor Liferenko
May 22 at 2:58
See EDIT in OP.
– Igor Liferenko
May 22 at 3:32
See EDIT in OP.
– Igor Liferenko
May 22 at 3:32
@IgorLiferenko It was not wrong; I reworded anyway. About the six files, I don't know what to say: I'm not going to download them from an unknown repository
– egreg
May 22 at 8:43
@IgorLiferenko It was not wrong; I reworded anyway. About the six files, I don't know what to say: I'm not going to download them from an unknown repository
– egreg
May 22 at 8:43
add a comment |
Why TeX skips end-of-line after ERROR from the first time, but after a only from the second time?
TeX removes any spaces from the end of the line (some older implementations erroneously removed tabs as well) and then if endlinechar
is in the 0-255 range that character is added to the end of the line) in plain and latex that defaults to 13 (control M).
As ^M
has catcode 5, it normally acts as space so is gobbled while tokenizing ERROR
and which explains the different behaviour for the first two lines.
if you delete 2 tokens then end
works
! Undefined control sequence.
l.1 ERROR
? 1
l.2 a
? 1
l.2 a
?
)
*end
No pages of output.
using pdftex
here.
with three 1 you get
! Undefined control sequence.
l.1 ERROR
? 1
l.2 a
? 1
l.2 a
? 1
)
*end
<*> end
because the final 1 gobbles the next token but that is the end
that you are about to add....
The difference is because you do not type1
after the last?
.
– Igor Liferenko
May 21 at 9:38
@IgorLiferenko yes I just updated the answer
– David Carlisle
May 21 at 9:39
add a comment |
Why TeX skips end-of-line after ERROR from the first time, but after a only from the second time?
TeX removes any spaces from the end of the line (some older implementations erroneously removed tabs as well) and then if endlinechar
is in the 0-255 range that character is added to the end of the line) in plain and latex that defaults to 13 (control M).
As ^M
has catcode 5, it normally acts as space so is gobbled while tokenizing ERROR
and which explains the different behaviour for the first two lines.
if you delete 2 tokens then end
works
! Undefined control sequence.
l.1 ERROR
? 1
l.2 a
? 1
l.2 a
?
)
*end
No pages of output.
using pdftex
here.
with three 1 you get
! Undefined control sequence.
l.1 ERROR
? 1
l.2 a
? 1
l.2 a
? 1
)
*end
<*> end
because the final 1 gobbles the next token but that is the end
that you are about to add....
The difference is because you do not type1
after the last?
.
– Igor Liferenko
May 21 at 9:38
@IgorLiferenko yes I just updated the answer
– David Carlisle
May 21 at 9:39
add a comment |
Why TeX skips end-of-line after ERROR from the first time, but after a only from the second time?
TeX removes any spaces from the end of the line (some older implementations erroneously removed tabs as well) and then if endlinechar
is in the 0-255 range that character is added to the end of the line) in plain and latex that defaults to 13 (control M).
As ^M
has catcode 5, it normally acts as space so is gobbled while tokenizing ERROR
and which explains the different behaviour for the first two lines.
if you delete 2 tokens then end
works
! Undefined control sequence.
l.1 ERROR
? 1
l.2 a
? 1
l.2 a
?
)
*end
No pages of output.
using pdftex
here.
with three 1 you get
! Undefined control sequence.
l.1 ERROR
? 1
l.2 a
? 1
l.2 a
? 1
)
*end
<*> end
because the final 1 gobbles the next token but that is the end
that you are about to add....
Why TeX skips end-of-line after ERROR from the first time, but after a only from the second time?
TeX removes any spaces from the end of the line (some older implementations erroneously removed tabs as well) and then if endlinechar
is in the 0-255 range that character is added to the end of the line) in plain and latex that defaults to 13 (control M).
As ^M
has catcode 5, it normally acts as space so is gobbled while tokenizing ERROR
and which explains the different behaviour for the first two lines.
if you delete 2 tokens then end
works
! Undefined control sequence.
l.1 ERROR
? 1
l.2 a
? 1
l.2 a
?
)
*end
No pages of output.
using pdftex
here.
with three 1 you get
! Undefined control sequence.
l.1 ERROR
? 1
l.2 a
? 1
l.2 a
? 1
)
*end
<*> end
because the final 1 gobbles the next token but that is the end
that you are about to add....
edited May 21 at 9:44
answered May 21 at 9:35
David CarlisleDavid Carlisle
506k4211531905
506k4211531905
The difference is because you do not type1
after the last?
.
– Igor Liferenko
May 21 at 9:38
@IgorLiferenko yes I just updated the answer
– David Carlisle
May 21 at 9:39
add a comment |
The difference is because you do not type1
after the last?
.
– Igor Liferenko
May 21 at 9:38
@IgorLiferenko yes I just updated the answer
– David Carlisle
May 21 at 9:39
The difference is because you do not type
1
after the last ?
.– Igor Liferenko
May 21 at 9:38
The difference is because you do not type
1
after the last ?
.– Igor Liferenko
May 21 at 9:38
@IgorLiferenko yes I just updated the answer
– David Carlisle
May 21 at 9:39
@IgorLiferenko yes I just updated the answer
– David Carlisle
May 21 at 9:39
add a comment |
Thanks for contributing an answer to TeX - LaTeX Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2ftex.stackexchange.com%2fquestions%2f491891%2fwhat-tokens-are-in-the-end-of-line%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
correction) in file 2) space is skipped, it's ^^M which is not, in file 3) space not skipped it's ^^M. file 6) is same as 2 space skkiped but ^^M not.
– touhami
May 22 at 23:07