Pavel Begunkov f6dc5a3619 io_uring: fix mixed cqe overflow handling ...

I started to see zcrx data corruptions. That turned out to be due
to CQ tail pointing to a stale entry which happened to be from
a zcrx request. I.e. the tail is incremented without the CQE
memory being changed.

The culprit is __io_cqring_overflow_flush() passing "cqe32=true"
to io_get_cqe_overflow() for non-mixed CQE32 setups, which only
expects it to be set for mixed 32B CQEs and not for SETUP_CQE32.

The fix is slightly hacky, long term it's better to unify mixed and
CQE32 handling.

Fixes: e26dca67fd ("io_uring: add support for IORING_SETUP_CQE_MIXED")
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

2025-11-25 07:03:45 -07:00

..

advise.c

io_uring: finish IOU_OK -> IOU_COMPLETE transition

2025-05-21 08:41:16 -06:00

advise.h

io_uring: split out fadvise/madvise operations

2022-07-24 18:39:11 -06:00

alloc_cache.c

io_uring: add alloc_cache.c

2025-01-28 15:10:40 -07:00

alloc_cache.h

io_uring/net: convert to struct iou_vec

2025-03-07 13:41:08 -07:00

cancel.c

io_uring: don't include filetable.h in io_uring.h

2025-09-08 13:20:46 -06:00

cancel.h

io_uring/cancel: add generic cancel helper

2025-02-17 05:34:45 -07:00

cmd_net.c

io_uring/cmd_net: fix wrong argument types for skb_queue_splice()

2025-11-20 11:40:15 -07:00

epoll.c

io_uring: finish IOU_OK -> IOU_COMPLETE transition

2025-05-21 08:41:16 -06:00

epoll.h

io_uring/epoll: add support for IORING_OP_EPOLL_WAIT

2025-02-20 07:59:56 -07:00

eventfd.c

io_uring/eventfd: open code io_eventfd_grab()

2025-04-24 08:33:54 -06:00

eventfd.h

io_uring/eventfd: dedup signalling helpers

2025-04-24 08:33:54 -06:00

fdinfo.c

io_uring/sqpoll: switch away from getrusage() for CPU accounting

2025-10-22 10:51:20 -06:00

fdinfo.h

io_uring: move fdinfo helpers to its own file

2022-07-24 18:39:12 -06:00

filetable.c

io_uring: correct __must_hold annotation in io_install_fixed_file

2025-10-23 07:25:07 -06:00

filetable.h

io_uring/rsrc: pass 'struct io_ring_ctx' reference to rsrc helpers

2024-11-07 15:24:33 -07:00

fs.c

io_uring: finish IOU_OK -> IOU_COMPLETE transition

2025-05-21 08:41:16 -06:00

fs.h

io_uring: split out filesystem related operations

2022-07-24 18:39:11 -06:00

futex.c

io_uring: add async data clear/free helpers

2025-08-27 11:24:25 -06:00

futex.h

io_uring: move cancelations to be io_uring_task based

2024-11-06 13:55:38 -07:00

io_uring.c

io_uring: fix mixed cqe overflow handling

2025-11-25 07:03:45 -07:00

io_uring.h

Merge tag 'for-6.18/io_uring-20250929' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux

2025-10-02 09:56:23 -07:00

io-wq.c

io_uring/io-wq: fix max_workers breakage and nr_workers underflow

2025-09-15 10:46:13 -06:00

io-wq.h

io_uring/wq: avoid indirect do_work/free_work calls

2025-04-21 05:06:58 -06:00

kbuf.c

io_uring: fix buffer auto-commit for multishot uring_cmd

2025-10-23 19:41:31 -06:00

kbuf.h

io-uring: move struct io_br_sel into io_uring_types.h

2025-08-24 11:41:12 -06:00

Kconfig

io_uring: make zcrx depend on CONFIG_IO_URING

2025-03-31 07:07:44 -06:00

Makefile

io_uring: introduce io_uring querying

2025-09-08 08:06:37 -06:00

memmap.c

io_uring: fix types for region size calulation

2025-11-05 11:45:07 -07:00

memmap.h

io_uring: update parameter name in io_pin_pages function declaration

2025-05-09 07:58:22 -06:00

mock_file.c

io_uring/mock: add trivial poll handler

2025-07-02 08:10:26 -06:00

msg_ring.c

io_uring/msg_ring: kill alloc_cache for io_kiocb allocations

2025-09-18 13:59:15 -06:00

msg_ring.h

io_uring/msg_ring: Drop custom destructor

2024-12-27 10:08:21 -07:00

napi.c

net: use napi_id_valid helper

2025-02-17 16:43:04 -08:00

napi.h

io_uring/napi: add static napi tracking strategy

2024-11-06 13:55:38 -07:00

net.c

io_uring/net: ensure vectored buffer node import is tied to notification

2025-11-24 10:59:02 -07:00

net.h

io_uring/net: convert to struct iou_vec

2025-03-07 13:41:08 -07:00

nop.c

io_uring/nop: add support for IORING_SETUP_CQE_MIXED

2025-08-27 11:24:15 -06:00

nop.h

…

notif.c

Merge tag 'for-6.18/io_uring-20250929' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux

2025-10-02 09:56:23 -07:00

notif.h

io_uring/notif: implement notification stacking

2024-04-22 19:31:18 -06:00

opdef.c

io_uring: uring_cmd: add multishot support

2025-08-24 11:41:12 -06:00

opdef.h

io_uring: add struct io_cold_def->sqe_copy() method

2025-06-23 08:59:13 -06:00

openclose.c

io_uring: don't include filetable.h in io_uring.h

2025-09-08 13:20:46 -06:00

openclose.h

io_uring: add support for IORING_OP_PIPE

2025-04-21 05:06:58 -06:00

poll.c

Merge tag 'for-6.18/io_uring-20250929' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux

2025-10-02 09:56:23 -07:00

poll.h

io_uring/poll: introduce io_arm_apoll()

2025-06-23 09:00:12 -06:00

query.c

io_uring/query: return number of available queries

2025-11-10 14:59:35 -07:00

query.h

io_uring: introduce io_uring querying

2025-09-08 08:06:37 -06:00

refs.h

io_uring: always do atomic put from iowq

2025-04-03 08:31:57 -06:00

register.c

io_uring/zcrx: remove sync refill uapi

2025-11-03 08:55:58 -07:00

register.h

io_uring: temporarily disable registered waits

2024-11-15 09:58:34 -07:00

rsrc.c

io_uring/rsrc: don't use blk_rq_nr_phys_segments() as number of bvecs

2025-11-12 08:25:33 -07:00

rsrc.h

io_uring: export io_[un]account_mem

2025-07-16 16:23:28 -06:00

rw.c

io_uring/rw: ensure allocated iovec gets cleared for early failure

2025-11-10 14:59:08 -07:00

rw.h

io_uring/kbuf: pass bgid to io_buffer_select()

2025-04-21 05:06:58 -06:00

slist.h

io_uring: silence variable ‘prev’ set but not used warning

2023-03-09 10:10:58 -07:00

splice.c

io_uring: don't include filetable.h in io_uring.h

2025-09-08 13:20:46 -06:00

splice.h

io_uring/splice: open code 2nd direct file assignment

2024-10-29 13:43:28 -06:00

sqpoll.c

io_uring/sqpoll: be smarter on when to update the stime usage

2025-10-22 10:55:33 -06:00

sqpoll.h

io_uring/sqpoll: switch away from getrusage() for CPU accounting

2025-10-22 10:51:20 -06:00

statx.c

io_uring: finish IOU_OK -> IOU_COMPLETE transition

2025-05-21 08:41:16 -06:00

statx.h

io_uring: move statx handling to its own file

2022-07-24 18:39:11 -06:00

sync.c

io_uring: finish IOU_OK -> IOU_COMPLETE transition

2025-05-21 08:41:16 -06:00

sync.h

io_uring: split out fs related sync/fallocate functions

2022-07-24 18:39:11 -06:00

tctx.c

io_uring/wq: avoid indirect do_work/free_work calls

2025-04-21 05:06:58 -06:00

tctx.h

io_uring: simplify __io_uring_add_tctx_node

2022-10-07 12:25:30 -06:00

timeout.c

io_uring: include dying ring in task_work "should cancel" state

2025-09-18 10:24:50 -06:00

timeout.h

io_uring/timeout: don't export link t-out disarm helper

2025-05-06 10:11:23 -06:00

truncate.c

io_uring: finish IOU_OK -> IOU_COMPLETE transition

2025-05-21 08:41:16 -06:00

truncate.h

io_uring: add support for ftruncate

2024-02-09 09:04:39 -07:00

uring_cmd.c

Merge tag 'for-6.18/io_uring-20250929' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux

2025-10-02 09:56:23 -07:00

uring_cmd.h

io_uring/cmd: remove struct io_uring_cmd_data

2025-07-18 12:34:56 -06:00

waitid.c

io_uring: fix incorrect unlikely() usage in io_waitid_prep()

2025-10-20 09:22:09 -06:00

waitid.h

io_uring: move cancelations to be io_uring_task based

2024-11-06 13:55:38 -07:00

xattr.c

io_uring: finish IOU_OK -> IOU_COMPLETE transition

2025-05-21 08:41:16 -06:00

xattr.h

…

zcrx.c

io_uring/zcrx: remove sync refill uapi

2025-11-03 08:55:58 -07:00

zcrx.h

io_uring/zcrx: remove sync refill uapi

2025-11-03 08:55:58 -07:00