commit e81107d4c6bd098878af9796b24edc8d4a9524fd upstream.
My colleague ran into a program stall on a x86_64 server, where
n_tty_read() was waiting for data even if there was data in the buffer
in the pty. kernel stack for the stuck process looks like below.
#0 [ffff88303d107b58] __schedule at ffffffff815c4b20
#1 [ffff88303d107bd0] schedule at ffffffff815c513e
#2 [ffff88303d107bf0] schedule_timeout at ffffffff815c7818
#3 [ffff88303d107ca0] wait_woken at ffffffff81096bd2
#4 [ffff88303d107ce0] n_tty_read at ffffffff8136fa23
#5 [ffff88303d107dd0] tty_read at ffffffff81368013
#6 [ffff88303d107e20] __vfs_read at ffffffff811a3704
#7 [ffff88303d107ec0] vfs_read at ffffffff811a3a57
#8 [ffff88303d107f00] sys_read at ffffffff811a4306
#9 [ffff88303d107f50] entry_SYSCALL_64_fastpath at ffffffff815c86d7
There seems to be two problems causing this issue.
First, in drivers/tty/n_tty.c, __receive_buf() stores the data and
updates ldata->commit_head using smp_store_release() and then checks
the wait queue using waitqueue_active(). However, since there is no
memory barrier, __receive_buf() could return without calling
wake_up_interactive_poll(), and at the same time, n_tty_read() could
start to wait in wait_woken() as in the following chart.
__receive_buf() n_tty_read()
------------------------------------------------------------------------
if (waitqueue_active(&tty->read_wait))
/* Memory operations issued after the
RELEASE may be completed before the
RELEASE operation has completed */
add_wait_queue(&tty->read_wait, &wait);
...
if (!input_available_p(tty, 0)) {
smp_store_release(&ldata->commit_head,
ldata->read_head);
...
timeout = wait_woken(&wait,
TASK_INTERRUPTIBLE, timeout);
------------------------------------------------------------------------
The second problem is that n_tty_read() also lacks a memory barrier
call and could also cause __receive_buf() to return without calling
wake_up_interactive_poll(), and n_tty_read() to wait in wait_woken()
as in the chart below.
__receive_buf() n_tty_read()
------------------------------------------------------------------------
spin_lock_irqsave(&q->lock, flags);
/* from add_wait_queue() */
...
if (!input_available_p(tty, 0)) {
/* Memory operations issued after the
RELEASE may be completed before the
RELEASE operation has completed */
smp_store_release(&ldata->commit_head,
ldata->read_head);
if (waitqueue_active(&tty->read_wait))
__add_wait_queue(q, wait);
spin_unlock_irqrestore(&q->lock,flags);
/* from add_wait_queue() */
...
timeout = wait_woken(&wait,
TASK_INTERRUPTIBLE, timeout);
------------------------------------------------------------------------
There are also other places in drivers/tty/n_tty.c which have similar
calls to waitqueue_active(), so instead of adding many memory barrier
calls, this patch simply removes the call to waitqueue_active(),
leaving just wake_up*() behind.
This fixes both problems because, even though the memory access before
or after the spinlocks in both wake_up*() and add_wait_queue() can
sneak into the critical section, it cannot go past it and the critical
section assures that they will be serialized (please see "INTER-CPU
ACQUIRING BARRIER EFFECTS" in Documentation/memory-barriers.txt for a
better explanation). Moreover, the resulting code is much simpler.
Latency measurement using a ping-pong test over a pty doesn't show any
visible performance drop.
Signed-off-by: Kosuke Tatsukawa <tatsu@ab.jp.nec.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
[lizf: Backported to 3.4:
- adjust context
- s/wake_up_interruptible_poll/wake_up_interruptible/
- drop changes to __receive_buf() and n_tty_set_termios()]
Signed-off-by: Zefan Li <lizefan@huawei.com>
commit 4291086b1f upstream.
The tty atomic_write_lock does not provide an exclusion guarantee for
the tty driver if the termios settings are LECHO & !OPOST. And since
it is unexpected and not allowed to call TTY buffer helpers like
tty_insert_flip_string concurrently, this may lead to crashes when
concurrect writers call pty_write. In that case the following two
writers:
* the ECHOing from a workqueue and
* pty_write from the process
race and can overflow the corresponding TTY buffer like follows.
If we look into tty_insert_flip_string_fixed_flag, there is:
int space = __tty_buffer_request_room(port, goal, flags);
struct tty_buffer *tb = port->buf.tail;
...
memcpy(char_buf_ptr(tb, tb->used), chars, space);
...
tb->used += space;
so the race of the two can result in something like this:
A B
__tty_buffer_request_room
__tty_buffer_request_room
memcpy(buf(tb->used), ...)
tb->used += space;
memcpy(buf(tb->used), ...) ->BOOM
B's memcpy is past the tty_buffer due to the previous A's tb->used
increment.
Since the N_TTY line discipline input processing can output
concurrently with a tty write, obtain the N_TTY ldisc output_lock to
serialize echo output with normal tty writes. This ensures the tty
buffer helper tty_insert_flip_string is not called concurrently and
everything is fine.
Note that this is nicely reproducible by an ordinary user using
forkpty and some setup around that (raw termios + ECHO). And it is
present in kernels at least after commit
d945cb9cce (pty: Rework the pty layer to
use the normal buffering logic) in 2.6.31-rc3.
js: add more info to the commit log
js: switch to bool
js: lock unconditionally
js: lock only the tty->ops->write call
References: CVE-2014-0196
Reported-and-tested-by: Jiri Slaby <jslaby@suse.cz>
Signed-off-by: Peter Hurley <peter@hurleysoftware.com>
Signed-off-by: Jiri Slaby <jslaby@suse.cz>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
[bwh: Backported to 3.2: output_lock is a member of struct tty_struct]
Signed-off-by: Ben Hutchings <ben@decadent.org.uk>
commit dab73b4eb9 upstream.
I meet emacs hang in start if I do the operation below:
1: echo 3 > /proc/sys/vm/drop_caches
2: emacs BigFile
3: Press CTRL-S follow 2 immediately
Then emacs hang on, CTRL-Q can't resume, the terminal
hang on, you can do nothing with this terminal except
close it.
The reason is before emacs takeover control the tty,
we use CTRL-S to XOFF it. Then when emacs takeover the
control, it may don't use the flow-control, so emacs hang.
This patch fix it.
This patch will fix a kind of strange tty relation hang problem,
I believe I meet it with vim in ssh, and also see below bug report:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=465823
Signed-off-by: Wang YanQing <udknight@gmail.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
commit e9490e93c1 upstream.
Change the BUG_ON to WARN_ON and return in case of tty->read_buf==NULL. We want to track a
couple of long standing reports of this but at the same time we can avoid killing the box.
Signed-off-by: Stanislav Kozina <skozina@redhat.com>
Signed-off-by: Alan Cox <alan@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Remove all #inclusions of asm/system.h preparatory to splitting and killing
it. Performed with the following command:
perl -p -i -e 's!^#\s*include\s*<asm/system[.]h>.*\n!!' `grep -Irl '^#\s*include\s*<asm/system[.]h>' *`
Signed-off-by: David Howells <dhowells@redhat.com>
* 'tty-next' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty-2.6: (26 commits)
amba pl011: workaround for uart registers lockup
n_gsm: fix the wrong FCS handling
pch_uart: add missing comment about OKI ML7223
pch_uart: Add MSI support
tty: fix "IRQ45: nobody cared"
PTI feature to allow user to name and mark masterchannel request.
0 for o PTI Makefile bug.
tty: serial: samsung.c remove legacy PM code.
SERIAL: SC26xx: Fix link error.
serial: mrst_max3110: initialize waitqueue earlier
mrst_max3110: Change max missing message priority.
tty: s5pv210: Add delay loop on fifo reset function for UART
tty/serial: Fix XSCALE serial ports, e.g. ce4100
serial: bfin_5xx: fix off-by-one with resource size
drivers/tty: use printk_ratelimited() instead of printk_ratelimit()
tty: n_gsm: Added refcount usage to gsm_mux and gsm_dlci structs
tty: n_gsm: Add raw-ip support
tty: n_gsm: expose gsmtty device nodes at ldisc open time
pch_phub: Fix register miss-setting issue
serial: 8250, increase PASS_LIMIT
...
Unthrottling the TTY during close ends up enabling interrupts
on a device not on the active list, which will never have the
interrupts cleared. Doctor, it hurts when I do this.
>>> On 6/2/2011 at 01:56 AM, in message <20110601145608.3e586e16@bob.linux.org.uk>, Alan Cox <alan@linux.intel.com> wrote:
> On Wed, 01 Jun 2011 10:34:07 +1200
> "andrew mcgregor" <andrew.mcgregor@alliedtelesis.co.nz> wrote:
> > The LKML message
> > http://kerneltrap.org/mailarchive/linux-kernel/2010/2/25/4541847 from
> > February doesn't seem to have been resolved since. We struck the
> > issue, and the patch below (against 2.6.32) fixes it. Should I
> > supply a patch against 3.0.0rc?
>
> I think that would be sensible. I don't actually see how you hit it as
> the IRQ ought to be masked by then but it's certainly wrong for n_tty
> to be calling into check_unthrottle at that point.
>
> So yes please send a patch with a suitable Signed-off-by: line to
> linux-serial and cc GregKH <greg@kroah.com> as well.
>
> Alan
Signed-off-by: Andrew McGregor <andrew.mcgregor@alliedtelesis.co.nz>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
With the previous patch, we fixed another bug where read_buf was freed
while we still was in n_tty_read. We currently check whether read_buf
is NULL at the start of the function. Add one more check after we wake
up from waiting for input.
Signed-off-by: Jiri Slaby <jslaby@suse.cz>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
This reverts commit b1c43f82c5.
It was broken in so many ways, and results in random odd pty issues.
It re-introduced the buggy schedule_work() in flush_to_ldisc() that can
cause endless work-loops (see commit a5660b41af: "tty: fix endless
work loop when the buffer fills up").
It also used an "unsigned int" return value fo the ->receive_buf()
function, but then made multiple functions return a negative error code,
and didn't actually check for the error in the caller.
And it didn't actually work at all. BenH bisected down odd tty behavior
to it:
"It looks like the patch is causing some major malfunctions of the X
server for me, possibly related to PTYs. For example, cat'ing a
large file in a gnome terminal hangs the kernel for -minutes- in a
loop of what looks like flush_to_ldisc/workqueue code, (some ftrace
data in the quoted bits further down).
...
Some more data: It -looks- like what happens is that the
flush_to_ldisc work queue entry constantly re-queues itself (because
the PTY is full ?) and the workqueue thread will basically loop
forver calling it without ever scheduling, thus starving the consumer
process that could have emptied the PTY."
which is pretty much exactly the problem we fixed in a5660b41af.
Milton Miller pointed out the 'unsigned int' issue.
Reported-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Reported-by: Milton Miller <miltonm@bga.com>
Cc: Stefan Bigler <stefan.bigler@keymile.com>
Cc: Toby Gray <toby.gray@realvnc.com>
Cc: Felipe Balbi <balbi@ti.com>
Cc: Greg Kroah-Hartman <gregkh@suse.de>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
it makes it simpler to keep track of the amount of
bytes received and simplifies how flush_to_ldisc counts
the remaining bytes. It also fixes a bug of lost bytes
on n_tty when flushing too many bytes via the USB
serial gadget driver.
Tested-by: Stefan Bigler <stefan.bigler@keymile.com>
Tested-by: Toby Gray <toby.gray@realvnc.com>
Signed-off-by: Felipe Balbi <balbi@ti.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Commit f23eb2b2b2 ('tty: stop using "delayed_work" in the tty layer')
ended up causing hung machines on UP with no preemption, because the
work routine to flip the buffer data to the ldisc would endlessly re-arm
itself if the destination buffer had filled up.
With the delayed work, that only caused a timer-driving polling of the
tty state every timer tick, but without the delay we just ended up with
basically a busy loop instead.
Stop the insane polling, and instead make the code that opens up the
receive room re-schedule the buffer flip work. That's what we should
have been doing anyway.
This same "poll for tty room" issue is almost certainly also the cause
of excessive kworker activity when idle reported by Dave Jones, who also
reported "flush_to_ldisc executing 2500 times a second" back in Nov 2010:
http://lkml.org/lkml/2010/11/30/592
which is that silly flushing done every timer tick. Wasting both power
and CPU for no good reason.
Reported-and-tested-by: Alexander Beregalov <a.beregalov@gmail.com>
Reported-and-tested-by: Sitsofe Wheeler <sitsofe@yahoo.com>
Cc: Greg KH <gregkh@suse.de>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Cc: Dave Jones <davej@redhat.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
The tty code should be in its own subdirectory and not in the char
driver with all of the cruft that is currently there.
Based on work done by Arnd Bergmann <arnd@arndb.de>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Cc: Jiri Slaby <jslaby@suse.cz>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
