Hi - I thought I'd give lsh a try, just to see how it compares to openssh... The client didn't work well on NetBSD, got a message like "unexpected EWOULDBLOCK" on each keystroke. Looked a bit deeper and found that stdin is set to O_NONBLOCK and a raw tty mode with c_cc[VMIN] > 1 and c_cc[VTIME] > 0. I'll append a little test program which does the same. I've tried it on 3 operating systems (Linux, NetBSD, Digital UNIX), and it behaves differently on each:
-on Linux, if a key is pressed, the read returns immediately with that one character -on NetBSD, the read returns with no data but EWOULDBLOCK -on D'UNIX, the poll() doesn't teturn before 4 keypresses are done; the read() returns these 4 characters
Indeed, in SUSv2's termios page is a sentence which says that if both O_NONBLOCK and VTIME>0 are set, the behaviour is more or less undefined.
I've solved my immediate problems by setting VMIN to 1 instead of 4 in unix_interact.c:do_make_raw(), but VTIME is still pointless, so I wouldn't call this a clean solution. (Don't know what liboop uses under the hood, but in case it does poll(), anything with VMIN>1 wouldn't work with D'Unix...)
best regards Matthias
Matthias Drochner M.Drochner@fz-juelich.de writes:
I thought I'd give lsh a try, just to see how it compares to openssh... The client didn't work well on NetBSD, got a message like "unexpected EWOULDBLOCK" on each keystroke.
I rely on the following behaviour of VMIN > 0 and VTIME > 0, as described by the glibc manual:
* Both TIME and MIN are nonzero.
In this case, TIME specifies how long to wait after each input character to see if more input arrives. After the first character received, `read' keeps waiting until either MIN bytes have arrived in all, or TIME elapses with no further input.
`read' always blocks until the first character arrives, even if TIME elapses first. `read' can return more than MIN characters if more than MIN happen to be in the queue.
and on the general principle that if I call read after that poll/select have said that a fd is readable, then read on that fd shouldn't return EWOULDBLOCK.
-on Linux, if a key is pressed, the read returns immediately with that one character -on NetBSD, the read returns with no data but EWOULDBLOCK -on D'UNIX, the poll() doesn't teturn before 4 keypresses are done; the read() returns these 4 characters
Interesting, all three seem broken to me, in one way or the other. I get the same result as you on my linux 2.4.18 laptop.
On netbsd, if you type quickly, will read return several characters at a time?
I also tried your test program on SunOS 5.9, where I had to add the definiton
#define cfmakeraw(ios) do { \ (ios)->c_iflag &= ~(IGNBRK|BRKINT|PARMRK|ISTRIP|INLCR|IGNCR|ICRNL|IXON); \ (ios)->c_oflag &= ~OPOST; \ (ios)->c_lflag &= ~(ECHO|ECHONL|ICANON|ISIG|IEXTEN); \ (ios)->c_cflag &= ~(CSIZE|PARENB); (ios)->c_cflag |= CS8; \ } while(0)
to make it compile. And then I increased VTIME to 10 to get a more noticable effect.
Then the test program works just the way it should: if I type less then 4 characters, I have to wait a while until read returns them all at the same time. If I type four characters quickly, they are returned as soon as I type the fourth one.
Indeed, in SUSv2's termios page is a sentence which says that if both O_NONBLOCK and VTIME>0 are set, the behaviour is more or less undefined.
Hmm, can you point me to the right place to read that?
(Don't know what liboop uses under the hood, but in case it does poll(), anything with VMIN>1 wouldn't work with D'Unix...)
Does it matter if one uses poll or select? If so, that may be a reason to prefer one over the other. (In general, I think the poll interface is nice and clean, but in practice the details differ so much between different systems that it is painful to use in portable code. select behaviour is much more uniform).
Then, possible workarounds. What happens if one jsut ignores the EWOULDBLOCK error? As long as the code doesn't degrade into a busy loop around poll/select and read, ignoring EWOULDBLOCK is harmless. This seems like the simplest solution. Untested patch below.
I don't quite like setting VMIN to 1, because that's a degradation for systems like Solaris where these things actually work. I guess it's possible to write a configure test that creates a pty pair and then tests behaviour, but that doesn't sound particularly fun.
Regards, /Niels
diff -u -a -p -r1.203 io.c --- src/io.c 25 Sep 2003 14:46:02 -0000 1.203 +++ src/io.c 15 Oct 2003 14:00:25 -0000 @@ -570,7 +570,7 @@ do_consuming_read(struct io_callback *c, case EINTR: break; case EWOULDBLOCK: - werror("io.c: read_consume: Unexpected EWOULDBLOCK\n"); + verbose("io.c: read_consume: Unexpected EWOULDBLOCK\n"); break; case EPIPE: /* FIXME: I don't understand why reading should return
nisse@lysator.liu.se said:
I rely on the following behaviour of VMIN > 0 and VTIME > 0, as described by the glibc manual: [...] `read' always blocks until the first character arrives
This obviously refers to the !O_NONBLOCK case...
if I call read after that poll/select have said that a fd is readable, then read on that fd shouldn't return EWOULDBLOCK.
Well the VMIN and VTIME stress the notion of "readable" somehow. One can take various approaches, or have different precedences: - If O_NONBLOCK, we can't sleep. Not even for the VTIME timeout? VTIME can be macroscopic, so we shouldn't probably. If so, VTIME is pointless. - A tty file descriptor might not be considered "readable" unless it fulfills the VMIN/VTIME conditions. Now there are some pitfalls: - a VTIME timeout begins at the time the read() call is done, so any poll() (or select(), ftm) isn't supposed to care about timing - also, the result of poll()/select() must not depend on setting of O_NONBLOCK (stated in some standard, maybe I'll find it...) - What does "readable" mean? "will return immediately with data" or "will not block indefinitely"? - One could assume: If a read() returns with less than VMIN chars, there was a gap of at least VTIME after the last. This might hold in presence of O_NONBLOCK or not...
For me, it seems pointless to make a sense of these contradicting requirements, the only thing which helps is to avoid certain situations. In particular, VTIME and O_NONBLOCK can't coexist, neither can VMIN and poll()/select().
On netbsd, if you type quickly, will read return several characters at a time?
I can't type quick enough to get more characters between the poll() and the read(). If I replace the poll() by a sleep(), the read() indeed returns as many characters.
I also tried your test program on SunOS 5.9 [...] then I increased VTIME to 10 to get a more noticable effect
Hmm - this is one of the gray areas... if there is a O_NONBLOCK, should the VTIME have an effect at all?
Indeed, in SUSv2's termios page is a sentence which says that if both O_NONBLOCK and VTIME>0 are set, the behaviour is more or less undefined.
Hmm, can you point me to the right place to read that?
http://www.opengroup.org/onlinepubs/007908799/xbd/termios.html The sentence I was referring to is in the "Non-canonical Mode Input Processing" chapter, beginning "The ISO POSIX-1 standard does not specify".
Does it matter if one uses poll or select? If so, that may be a reason to prefer one over the other.
I believe this is just a matter of the user interface. Both very likely base on the same kernel function. So they will behave identically.
(In general, I think the poll interface is nice and clean, but in practice the details differ so much between different systems that it is painful to use in portable code. select behaviour is much more uniform).
select() is older and supported on more systems, agreed. Nowadays I'd assume that poll() works on every system I'd consider worth working with...
What happens if one jsut ignores the EWOULDBLOCK error?
I didn't try yet, but I'd assume that no tty input is accepted unless 4 characters are typed in. (My failure report wasn't very precise, sorry. Actually, I didn't get just a single "EWOULDBLOCK" message after a keystroke, but lsh went into an endless loop printing these messages after I pressed a key.)
I don't quite like setting VMIN to 1
As said above, I _believe_ that VMIN>1 and poll()/select() together lead to nonportable code...
best regards Matthias
Matthias Drochner M.Drochner@fz-juelich.de writes:
nisse@lysator.liu.se said:
I rely on the following behaviour of VMIN > 0 and VTIME > 0, as described by the glibc manual: [...] `read' always blocks until the first character arrives
This obviously refers to the !O_NONBLOCK case...
Sure. I find it very unfortunate that the tty flags and the non-block flags aren't orthogonal.
- If O_NONBLOCK, we can't sleep.
Obviously, with O_NONBLOCK, read should not sleep. That's the point of O_NONBLOCK.
- A tty file descriptor might not be considered "readable" unless it fulfills the VMIN/VTIME conditions. Now there are some pitfalls:
- a VTIME timeout begins at the time the read() call is done,
Sure about that? I believe the timer should be started as soon as a character arrives to the tty driver, regardless of what system calls are made. That's also the way I read the spec you point to:
"In this case TIME serves as an inter-byte timer and is activated after the first byte is received. Since it is an inter-byte timer, it is reset after a byte is received."
In this context, I think "received" must mean that bytes arrive to the tty driver (via a the serial line or whatever).
- also, the result of poll()/select() must not depend on setting of O_NONBLOCK (stated in some standard, maybe I'll find it...)
Makes sense, too bad posix seems to not require this to be true for tty:s.
- What does "readable" mean? "will return immediately with data" or "will not block indefinitely"?
To me, "readable" mean that a plain vanilla ordinary blocking read on the fd would return immediately, with success (i.e. with data or an EOF indication) without putting my process to sleep. If a blocking read wouldn't return immediately due to the VTIME value, then the fd is *not* readable.
- One could assume: If a read() returns with less than VMIN chars, there was a gap of at least VTIME after the last. This might hold in presence of O_NONBLOCK or not...
Don't understand the relevance of this.
On netbsd, if you type quickly, will read return several characters at a time?
I can't type quick enough to get more characters between the poll() and the read().
And it doesn't matter if you increasing VTIME, I guess?
Hmm - this is one of the gray areas... if there is a O_NONBLOCK, should the VTIME have an effect at all?
Well, that's what you keep saying. I think the Solaris behaviour here makes perfect sense. The VMIN/VTIME are orthogonal to O_NONBLOCK, you can use one or the other, or both, and in all cases get sane behaviour. Linux and netbsd are appearantly broken (although still posix compliant on this issue), the question for lsh is how to best work around it.
http://www.opengroup.org/onlinepubs/007908799/xbd/termios.html The sentence I was referring to is in the "Non-canonical Mode Input Processing" chapter, beginning "The ISO POSIX-1 standard does not specify".
Thanks. This section says:
The ISO POSIX-1 standard does not specify whether the setting of O_NONBLOCK takes precedence over MIN or TIME settings. Therefore, if O_NONBLOCK is set, read() may return immediately, regardless of the setting of MIN or TIME. Also, if no data is available, read() may either return 0, or return -1 with errno set to [EAGAIN].
The second sentence says that the kernel can return immedately, without violating POSIX. The third says that the kernel can return immedately, *with 0 as a return value*.
*BLECH*
That basically means that an operating system can be perfectly posix compliant, without implementing a useful O_NONBLOCK at all (instead, O_NONBLOCK can be an alias for the older, totally braindead, O_NDELAY flag).
select() is older and supported on more systems, agreed. Nowadays I'd assume that poll() works on every system I'd consider worth working with...
Well, poll works on both linux and Solaris, but the way they set the various poll flags is very different. Don't know any details bsd's poll.
(My failure report wasn't very precise, sorry. Actually, I didn't get just a single "EWOULDBLOCK" message after a keystroke, but lsh went into an endless loop printing these messages after I pressed a key.)
That's worse, then just ignoring the error is not an option.
Exactly which version of netbsd are you using?
Regards, /Niels
PS. I think the O_NONBLOCK flag is a really bad design. I really wish kernel hackers would consider adding non-blocking syscalls like nonblock_read and nonblock_write, as suggested by djb at http://cr.yp.to/unix/nonblock.html).
[only a short reply, will be AFK until monday night]
nisse@lysator.liu.se said:
- a VTIME timeout begins at the time the read() call is done,
Sure about that? I believe the timer should be started as soon as a character arrives to the tty driver [...] "In this case TIME serves as an inter-byte timer and is activated after the first byte is received.
Only a few lines later: If the data is in the buffer at the time of the read(), the result will be as if the data has been received immediately after the read().
the result of poll()/select() must not depend on setting of O_NONBLOCK
Here is a reference: http://www.opengroup.org/onlinepubs/007908799/xsh/poll.html
If a blocking read wouldn't return immediately due to the VTIME value, then the fd is *not* readable.
If you take that and assume that the result of select()/poll() does not depend on O_NONBLOCK, then your code with VMIN==4 can't work as expected. The callback wouldn't be called until 4 characters are received. (D'UNIX appearently behaves this way, but neither Linux nor NetBSD)
The only way around this I can imagine is to have VMIN==1 most of the time; to get characters coalesced for efficiency, VMAX could be set to 4 for the time of the read() (and the O_NONBLOCK cleared). What a mess... (this would effectively mean that O_NONBLOCK is useless on the stdin tty descriptor)
poll works on both linux and Solaris, but the way they set the various poll flags is very different.
I wasn't aware of this...
Exactly which version of netbsd are you using?
Newest bleeding edge current on i386. I can do tests on older releases if needed, and perhaps also on other architectures (alpha, mips, m68k).
best regards Matthias
Matthias Drochner M.Drochner@fz-juelich.de writes:
Only a few lines later: If the data is in the buffer at the time of the read(), the result will be as if the data has been received immediately after the read().
Ok. My mental model of how this should works is this: The tty driver is one component, which operates the serial line, implements the various termios flags (including VMIN and VTIME), and makes characters available for reading via some kernel buffer. Then the read system call gets characters from that buffer, and the only difference between blocking and non-blocking read is what read does when the buffer is empty.
I guess the kernels that behave differently don't think of VMIN and VTIME as an internal feature of the tty driver, but somehow lets read and poll system calls see that the tty driver have an internal buffer of up to VMIN characters.
The only way around this I can imagine is to have VMIN==1 most of the time; to get characters coalesced for efficiency, VMAX could be set to 4 for the time of the read() (and the O_NONBLOCK cleared). What a mess...
I can see two ways to deal with this:
1. Write a configure test, and use VMIN > 1 only on systems that behave the way I think they should.
2. Spawn a separate process that reads stdin in blocking mode, and use a pipe to get the data to lsh.
You can actually try (2) already, use the somewhat obscure option --cvs-workaround or --cvs-workaround=ei to lsh. This hack exists because it's generally not very friendly to set one's stdio fds into non-blocking mode, and it is enable by default for stderr, which is the fd most commonly shared with other processes.
I've also created a bugzilla ticket for this, http://bugzilla.lysator.liu.se/show_bug.cgi?id=1175.
Regards, /Niels
I finally got arond to upgrading my lsh 1.2.X server tonight. I wanted to play with some of the cvs changes, but did not remember the repository. The CVS repository on the web (http://www.lysator.liu.se/~nisse/lsh/) page is listed as: "cvs -d :pserver:anonymous@cvs.lysator.liu.se co lsh"
that's incorrect, it should be listed as "cvs -d:pserver:anonymous@cvs.lysator.liu.se:/cvsroot/lsh co lsh"
I grabbed a copy of 1.5, d/l it on to a debian box (running testing, under a 2.4.21 kernel) run ./configure --disable-srp --disable-x11-forward --disable-ipv6 and make, make runs fine, but upon make install, I get
make[3]: Entering directory `/home/testbuild/lsh-1.5.2/src' false -l /home/testbuild/lsh-1.5.2/src/scm/false-compat.scm -e main -l /home/testbuild/lsh-1.5.2/src/scm/compiler.scm \ -s /home/testbuild/lsh-1.5.2/src/scm/gaba.scm <lsh.c >lsh.c.xT make[3]: *** [lsh.c.x] Error 1
I don't see a false-compat.scm in /src/scm, either in 1.5.2 or the cvs tree. digging deeper, It looks like this is telling me I'm missing or it can't find guile+slib or /scsh. (guile+slib are installed).
"hormel" hormel@www.badcode.org writes:
The CVS repository on the web (http://www.lysator.liu.se/~nisse/lsh/) page is listed as: "cvs -d :pserver:anonymous@cvs.lysator.liu.se co lsh" that's incorrect, it should be listed as "cvs -d:pserver:anonymous@cvs.lysator.liu.se:/cvsroot/lsh co lsh"
Thanks, fixed now.
and make, make runs fine, but upon make install, I get
make[3]: Entering directory `/home/testbuild/lsh-1.5.2/src' false -l /home/testbuild/lsh-1.5.2/src/scm/false-compat.scm -e main -l /home/testbuild/lsh-1.5.2/src/scm/compiler.scm \ -s /home/testbuild/lsh-1.5.2/src/scm/gaba.scm <lsh.c >lsh.c.xT make[3]: *** [lsh.c.x] Error 1
There seems to be several problems here. First, it seems that before you ran make, you already had automatically generated files like lsh.c.x lying around (if not, you would have got compilation errors from the first make), but you didn't have the corresponding dependency files, like .deps/lsh.Po (if you did have that, make should have tried to rebuild the lsh.c.x file the first time you ran make).
It looks like this is telling me I'm missing or it can't find guile+slib or /scsh. (guile+slib are installed).
Right, you must have guile or scsh installed. The configure script looks for them. So there might be a configure bug, check config.log to see what happened.
And at last, the recommended way to build lsh from cvs is to check out the source, then write
./.bootstrap && configure --your-favourite-flags && make bootstrap make
The first line makes sure that you have all the automatically generated files which are included in the lsh distributions, but not in CVS. The second line compiles the source, and creates dependency files which are used the next time you run make.
Regards, /Niels