User Interrupt#

Intel UINTR#

Intro.#

User interrupts are delivered to software operating in 64-bit mode with CPL=3 without any change to segmentation state. Different user interrupts are distinguished by a 6-bit user-interrupt vector. Based on x86, the UINTR state is configured by new supervisor-managed state, including new MSRs.

UPID : user posted-interrupt descriptor. User interrupts for an OS-managed thread can be posted in the UPID associated with that thread after receipt of an ordinary interrupt called a user-interrupt notification.

  • Outstanding notification (0): For one or more user interrupts in PIR (as |pir_bits).

  • Suppress notification (1): If set, agents should not send notifications when posting user interrupts in this descriptor.

  • Notification vector (23:16): Used by SENDUIPI.

  • Notification destination (63:32): Target physical APIC ID.

  • Posted-interrupt requests (PIR) (127:64): One bit for each user-interrupt vector. See UIRR.

Setting CR4[25] (as CR4.UINTR) enables user-interrupt delivery, user-interrupt notification identification and the user-interrupt instructions (UIRET, SENDUIPI).

User-interrupt State :

  • UIRR : If UIRR[i] = 1, a user interrupt with vector i is requesting service.

  • UIF : If UIF = 0, user-interrupt delivery is blocked; if UIF = 1, user interrupts may be delivered. User-interrupt delivery clears UIF, and the new UIRET instruction sets it.

  • UIHANDLER : The linear address of the user-interrupt handler.

  • UISTACKADJUST : This value controls adjustment to the stack pointer prior to user-interrupt delivery.

  • UINV : The vector of those ordinary interrupts that are treated as user-interrupt notifications.

  • UPIDADDR : The linear address of the UPID that the logical processor consults upon receiving an ordinary interrupt with vector UINV.

  • UITTADDR : The linear address of user-interrupt target table, which the logical processor consults when software invokes the SENDUIPI instruction.

  • UITTSZ : The highest index of a valid entry in the UITT.

User-interrupt MSRs :

  • IA32_UINTR_RR -> UIRR

  • IA32_UINTR_HANDLER -> UIHANDLER

  • IA32_UINTR_STACKADJUST -> UISTACKADJUST

  • IA32_UINTR_MISC -> {24’b0, UINV[7:0], UITTSZ[31:0]}

  • IA32_UINTR_PD -> UPIDADDR

  • IA32_UINTR_TT -> UITTADDR

User-interrupt notification identification : How to identify an ordinary interrupt as user-interrupt notification? The local APIC is acknowledged; move to the next step if the processor get an interrupt vector V = UINV. The processor writes zero to the EOI register in the local APIC; this dismiss the interrupt with vector V = UINV from the local APIC.

APIC

Advanced Programmable Interrupt Controller. Intended to solve interrupt routing efficiency issues in computer systems. In a APIC-based system, each CPU is made of a core and a local APIC. In addition, there is an I/O APIC that is part of the chipset and provides multi-processor interrupt management. Each interrupt pin is individually programmable as either edge or level triggered. The interrupt vector and interrupt steering information can be specified per interrupt.

User-interrupt notification processing :

  • The logical processor clears the outstanding-notification bit in the UPID. This is done atomically so as to leave the remainder of the descriptor unmodified.

  • The logical processor reads PIR into a temporary register and writes all zeros to PIR. This is done atomically so as to ensure that each bit cleared is set in the temporary register.

  • If any bit is set in the temporary register, the logical processor sets in UIRR each bit corresponding to a bit set in the temporary register and recognize a pending user interrupt.

Impl. in Linux#

UPID and UITT are managed by kernel.

Instructions:

  • SENDUIPI <index>: send a user IPI to a target task based on the UITT index.

  • CLUI: Mask user interrupts by clearing UIF.

  • STUI: Unmask user interrupts by setting UIF.

  • TESTUI: Test current value of UIF.

  • UIRET: return from a user interrupt handler.

UITT entry:

  • (0): V, a valid bit

  • (7:1): reserved

  • (15:8): UV, the user-interrupt vector (15:14 is wired to zero)

  • (63:16): reserved

  • (127:64): UPIDADDR (64-byte aligned, so 69:64 is wired to zero)

See implementations in uintr_linux .

 1/* Vector for User interrupt notifications */
 2#define UINTR_NOTIFICATION_VECTOR       0xec
 3#define UINTR_KERNEL_VECTOR                 0xeb
 4
 5int do_uintr_register_handler(u64 handler)
 6{
 7    struct uintr_receiver *ui_recv;
 8    struct uintr_upid *upid;
 9    struct task_struct *t = current;
10    struct fpu *fpu = &t->thread.fpu;
11    u64 misc_msr;
12    int cpu;
13
14    if (is_uintr_receiver(t))
15        return -EBUSY;
16
17    /* Alloc new uintr receiver with UPID */
18    ui_recv = kzalloc(sizeof(*ui_recv), GFP_KERNEL);
19    ui_recv->upid_ctx = alloc_upid();
20
21    /*
22     * UPID and ui_recv will be referenced during context switch. Need to
23     * disable preemption while modifying the MSRs, UPID and ui_recv thread
24     * struct.
25     */
26    fpregs_lock();
27
28    /* Find the receiver and set the notification vector and destination */
29    cpu = smp_processor_id();
30    upid = ui_recv->upid_ctx->upid;
31    upid->nc.nv = UINTR_NOTIFICATION_VECTOR;
32    upid->nc.ndst = cpu_to_ndst(cpu);
33
34    t->thread.ui_recv = ui_recv;
35
36    if (fpregs_state_valid(fpu, cpu)) {
37        wrmsrl(MSR_IA32_UINTR_HANDLER, handler);
38        wrmsrl(MSR_IA32_UINTR_PD, (u64)ui_recv->upid_ctx->upid);
39
40        /* Set value as size of ABI redzone */
41        wrmsrl(MSR_IA32_UINTR_STACKADJUST, 128);
42
43        /* Modify only the relevant bits of the MISC MSR */
44        rdmsrl(MSR_IA32_UINTR_MISC, misc_msr);
45        misc_msr |= (u64)UINTR_NOTIFICATION_VECTOR << 32;
46        wrmsrl(MSR_IA32_UINTR_MISC, misc_msr);
47    } else {
48        struct xregs_state *xsave;
49        struct uintr_state *p;
50
51        xsave = &fpu->state.xsave;
52        xsave->header.xfeatures |= XFEATURE_MASK_UINTR;
53        p = get_xsave_addr(&fpu->state.xsave, XFEATURE_UINTR);
54        if (p) {
55            p->handler = handler;
56            p->upid_addr = (u64)ui_recv->upid_ctx->upid;
57            p->stack_adjust = 128;
58            p->uinv = UINTR_NOTIFICATION_VECTOR;
59        }
60    }
61
62    fpregs_unlock();
63
64    return 0;
65}
66
67int do_uintr_register_vector(struct uintr_receiver_info *r_info)
68{
69    struct uintr_receiver *ui_recv;
70    struct task_struct *t = current;
71    /*
72    * A vector should only be registered by a task that
73    * has an interrupt handler registered.
74    */
75    if (!is_uintr_receiver(t)) return -EINVAL;
76    if (r_info->uvec >= UINTR_MAX_UVEC_NR) return -ENOSPC;
77    ui_recv = t->thread.ui_recv;
78    if (ui_recv->uvec_mask & BIT_ULL(r_info->uvec)) return -EBUSY;
79    ui_recv->uvec_mask |= BIT_ULL(r_info->uvec);
80    r_info->upid_ctx = get_upid_ref(ui_recv->upid_ctx);
81    return 0;
82}

int do_uintr_register_handler(u64 handler) allocates new user-interrupt context and UPID for current task. It disables preemption (the ability of the operating system to preempt or stop a currently scheduled task in favour of a higher priority task) during modifying task states. Then it sets the destination as target APIC id. It writes MSRs finally (or xsave states) to activate hardware process.

 1int do_uintr_register_sender(struct uintr_receiver_info *r_info,
 2                         struct uintr_sender_info *s_info)
 3{
 4    struct uintr_uitt_entry *uitte = NULL;
 5    struct uintr_sender *ui_send;
 6    struct task_struct *t = current;
 7    unsigned long flags;
 8    int entry;
 9    int ret;
10
11    /*
12    * Only a static check. Receiver could exit anytime after this check.
13    * This check only prevents connections using uintr_fd after the
14    * receiver has already exited/unregistered.
15    */
16    if (!uintr_is_receiver_active(r_info))
17        return -ESHUTDOWN;
18
19    if (is_uintr_sender(t)) {
20        entry = find_first_zero_bit((unsigned long *)t->thread.ui_send->uitt_mask,
21                        UINTR_MAX_UITT_NR);
22        if (entry >= UINTR_MAX_UITT_NR)
23            return -ENOSPC;
24    } else {
25        BUILD_BUG_ON(UINTR_MAX_UITT_NR < 1);
26        entry = 0;
27        ret = init_uitt();
28        if (ret)
29            return ret;
30    }
31
32    ui_send = t->thread.ui_send;
33
34    set_bit(entry, (unsigned long *)ui_send->uitt_mask);
35
36    spin_lock_irqsave(&ui_send->uitt_ctx->uitt_lock, flags);
37    uitte = &ui_send->uitt_ctx->uitt[entry];
38    pr_debug("send: sender=%d receiver=%d UITTE entry %d address %px\n",
39        current->pid, r_info->upid_ctx->task->pid, entry, uitte);
40
41    uitte->user_vec = r_info->uvec;
42    uitte->target_upid_addr = (u64)r_info->upid_ctx->upid;
43    uitte->valid = 1;
44    spin_unlock_irqrestore(&ui_send->uitt_ctx->uitt_lock, flags);
45
46    s_info->r_upid_ctx = get_upid_ref(r_info->upid_ctx);
47    s_info->uitt_ctx = get_uitt_ref(ui_send->uitt_ctx);
48    s_info->task = get_task_struct(current);
49    s_info->uitt_index = entry;
50
51    return 0;
52}

 1int uintr_receiver_wait(void)
 2{
 3    struct uintr_upid_ctx *upid_ctx;
 4    unsigned long flags;
 5
 6    if (!is_uintr_receiver(current))
 7        return -EOPNOTSUPP;
 8
 9    upid_ctx = current->thread.ui_recv->upid_ctx;
10    upid_ctx->upid->nc.nv = UINTR_KERNEL_VECTOR;
11    upid_ctx->waiting = true;
12    spin_lock_irqsave(&uintr_wait_lock, flags);
13    list_add(&upid_ctx->node, &uintr_wait_list);
14    spin_unlock_irqrestore(&uintr_wait_lock, flags);
15
16    set_current_state(TASK_INTERRUPTIBLE);
17    schedule();
18
19    return -EINTR;
20}
21
22/*
23 * Runs in interrupt context.
24 * Scan through all UPIDs to check if any interrupt is on going.
25 */
26void uintr_wake_up_process(void)
27{
28    struct uintr_upid_ctx *upid_ctx, *tmp;
29    unsigned long flags;
30
31    spin_lock_irqsave(&uintr_wait_lock, flags);
32    list_for_each_entry_safe(upid_ctx, tmp, &uintr_wait_list, node) {
33            if (test_bit(UPID_ON, (unsigned long *)&upid_ctx->upid->nc.status)) {
34                    set_bit(UPID_SN, (unsigned long *)&upid_ctx->upid->nc.status);
35                    upid_ctx->upid->nc.nv = UINTR_NOTIFICATION_VECTOR;
36                    upid_ctx->waiting = false;
37                    wake_up_process(upid_ctx->task);
38                    list_del(&upid_ctx->node);
39            }
40    }
41    spin_unlock_irqrestore(&uintr_wait_lock, flags);
42}