Writing Solaris Device Driver: Interrupt Handlers

Interrupt Handlers

This chapter describes mechanisms for handling interrupts, such as registering, servicing, and removing interrupts. This chapter provides information on the following subjects:

• Interrupt Handler Overview

• Device Interrupts

• Registering Interrupts

• Interrupt Handler Responsibilities

• Handling High-Level Interrupts

Interrupt Handler Overview

An interrupt is a hardware signal from a device to a CPU. An interrupt tells the CPU that the device needs attention and that the CPU should stop any current activity and respond to the device. If a CPU is available, that is, not performing a task with higher priority, the CPU suspends the current thread. The CPU then invokes the interrupt handler for that device. The job of the interrupt handler is to service the device and stop the device from interrupting. Once the handler returns, the CPU resumes the activity from before the interrupt occurred.

The DDI/DKI provides interfaces for registering and servicing interrupts. New interfaces have been added for performing such tasks as:

• Determining interrupt type and registration requirements

• Masking interrupts

• Getting interrupt pending information

• Getting and setting priority information

Device Interrupts

I/O buses implement interrupts in two common ways: vectored and polled. Both methods commonly supply a bus-interrupt priority level. Vectored devices also supply an interrupt vector. Polled devices do not supply interrupt vectors.

To keep up with changing bus technologies, the Solaris OS has been enhanced to accommodate both newer types of interrupts and more traditional interrupts that have been in use for many years. Specifically, the operating system now recognizes three types of interrupts:

• Legacy interrupts – Legacy or fixed interrupts refer to interrupts that use older bus technologies. With these technologies, interrupts are signalled by using one or more external pins that are wired “out-of-band,” that is, separately from the main lines of the bus. Newer bus technologies like PCI Express maintain software compatibility by emulating legacy interrupts through in-band mechanisms. These emulated interrupts are treated as legacy interrupts by the host OS.

• Message-signalled interrupts – Instead of using pins, message-signalled interrupts (MSI) are in-band messages and can target addresses in the host bridge. (See PCI Local Bus for more information on host bridges.) MSIs can send data along with the interrupt message. Each MSI is unshared so that an MSI that is assigned to a device is guaranteed to be unique within the system. A PCI function can request up to 32 MSI messages.

• Extended message-signalled interrupts – Extended message-signalled interrupts (MSI-X) are an enhanced version of MSIs. MSI-X interrupts have these added advantages:

  • Supports 2048 messages rather than 32 messages

  • Supports independent message address and message data for each message

  • Supports per-message masking

  • Enables more flexibility when software allocates fewer vectors than hardware requests. The software can reuse the same MSI-X address and data in multiple MSI-X slots.

---

Note –

Some newer bus technologies such as PCI Express require MSIs but can accommodate legacy interrupts by using INTx emulation. The INTx emulation is for compatibility purposes, but is not considered good practice.

---

High-Level Interrupts

A bus prioritizes a device interrupt at a bus-interrupt level. The bus interrupt level is then mapped to a processor-interrupt level. A bus interrupt level that maps to a CPU interrupt priority above the scheduler priority level is called a high-level interrupt. High-level interrupt handlers are restricted to calling the following DDI interfaces:

• mutex_enter(9F) and mutex_exit(9F) on a mutex that is initialized with an iblock cookie associated with the high-level interrupt

• ddi_intr_trigger_softint(9F)

• The ddi_getX/ddi_putX families of routines

A bus-interrupt level by itself does not determine whether a device interrupts at a high level: a given bus-interrupt level can map to a high-level interrupt on one platform, but map to an ordinary interrupt on another platform.

Although the driver can choose whether to support devices that have high-level interrupts, the driver is always required to check the interrupt level. The function ddi_intr_get_hilevel(9F) returns the minimum priority level for a high-level interrupt. This value can be compared against other priority values from ddi_intr_get_pri(9F) to determine if a value is actually a high-level interrupt.

Legacy Interrupts

The only information that the system has about a device interrupt is the priority level for the bus interrupt, for example, the IPL on an SBus in a SPARC machine and the interrupt request number, for example, the IRQ on an ISA bus in an x86 machine.

When an interrupt handler is registered, the system adds the handler to a list of potential interrupt handlers for each IPL or IRQ. When the interrupt occurs, the system must determine which device out of all devices that are associated with a given IPL or IRQ, actually caused the interrupt. The system calls all the interrupt handlers for the designated IPL or IRQ until one handler claims the interrupt.

The following buses are capable of supporting polled interrupts:

• SBus

• ISA

• PCI

Standard and Extended Message-Signalled Interrupts

Both standard (MSI) and extended (MSI-X) message-signalled interrupts are implemented as in-band messages. A message-signalled interrupt is posted as a write with an address and value that are specified by the software.

MSI Interrupts

Conventional PCI specifications include optional support for Message Signaled Interrupts (MSI). An MSI interrupt is an in-band message that is implemented as a posted write. The address and the data for the MSI are specified by software and are specific to the host bridge. Because the messages are in-band, the receipt of the message can be used to “push” data that is associated with the interrupt. By definition, MSI interrupts are unshared. Each MSI message that is assigned to a device is guaranteed to be a unique message in the system. PCI functions can request between 1 and 32 MSI messages, in powers of two. Note that the system software can allocate fewer MSI messages to a function than the function requested. The host bridge can be limited in the number of unique MSI messages that are allocated for devices.

MSI-X Interrupts

MSI-X interrupts are enhanced versions of MSI interrupts that have the same features as MSI interrupts with the following key differences:

• A maximum of 2048 MSI-X interrupt vectors are supported per device.

• Address and data entries are unique per interrupt vector.

• MSI-X supports per function masking and per vector masking.

With MSI-X interrupts, an unallocated interrupt vector of a device can use a previously added or initialized MSI-X interrupt vector to share the same vector address, vector data, interrupt handler, and handler arguments. Use the ddi_intr_dup_handler(9F) function to alias the resources provided by the Solaris OS to the unallocated interrupt vectors on an associated device. For example, if 2 MSI-X interrupts are allocated to a driver and 32 interrupts are supported on the device, then the driver can use ddi_intr_dup_handler() to alias the 2 interrupts it received to the 30 additional interrupts on the device.

The ddi_intr_dup_handler() function can duplicate interrupts that were added with ddi_intr_add_handler(9F) or initialized with ddi_intr_enable(9F).

A duplicated interrupt is disabled initially. Use ddi_intr_enable() to enable the duplicated interrupt. You cannot remove the original MSI-X interrupt handler until all duplicated interrupt handlers that are associated with this original interrupt handler are removed. To remove a duplicated interrupt handler, first call ddi_intr_disable(9F), and then call ddi_intr_free(9F). When all duplicated interrupt handlers that are associated with this original interrupt handler are removed, then you can use ddi_intr_remove_handler(9F) to remove the original MSI-X interrupt handler. See the ddi_intr_dup_handler(9F) man page for examples.

Software Interrupts

The Solaris DDI/DKI supports software interrupts, also known as soft interrupts. Soft interrupts are initiated by software rather than by a hardware device. Handlers for these interrupts must also be added to and removed from the system. Soft interrupt handlers run in interrupt context and therefore can be used to do many of the tasks that belong to an interrupt handler.

Hardware interrupt handlers must perform their tasks quickly, because the handlers might have to suspend other system activity while doing these tasks. This requirement is particularly true for high-level interrupt handlers, which operate at priority levels greater than the priority level of the system scheduler. High-level interrupt handlers mask the operations of all lower-priority interrupts, including the interrupt operations of the system clock. Consequently, the interrupt handler must avoid involvement in activities that might cause it to sleep, such as acquiring a mutex.

If the handler sleeps, then the system might hang because the clock is masked and incapable of scheduling the sleeping thread. For this reason, high-level interrupt handlers normally perform a minimum amount of work at high-priority levels and delegate other tasks to software interrupts, which run below the priority level of the high-level interrupt handler. Because software interrupt handlers run below the priority level of the system scheduler, software interrupt handlers can do the work that the high-level interrupt handler was incapable of doing.

DDI Interrupt Function Changes

The Solaris OS now provides a new framework for registering and unregistering interrupts, along with support for Message Signalled Interrupts (MSIs). New management interfaces let you manipulate priorities, capabilities, and interrupt masking, and obtain pending information.

This section shows the changes that have been made to the interrupt functions since the Solaris 10 OS FCS release.

Interrupt Capability Functions

The following functions are the preferred methods for obtaining interrupt information:

ddi_intr_get_navail(9F)

Returns the number of interrupts available for a particular hardware device and given interrupt type.

ddi_intr_get_nintrs(9F)

Gets the number of interrupts that the device supports for the given interrupt type.

ddi_intr_get_supported_types(9F)

Returns the hardware interrupt types that are supported by both the device and the host.

ddi_intr_get_cap(9F)

Returns interrupt capability flags for the specified interrupt.

Interrupt Initialization and Destruction Functions

The following functions are used in the process of creating and removing interrupts:

ddi_intr_alloc(9F)

Allocates system resources and interrupt vectors for the specified type of interrupt.

ddi_intr_free(9F)

Releases the system resources and interrupt vectors for a specified interrupt handle.

ddi_intr_set_cap(9F)

Sets the capability of the specified interrupt though the use of the DDI_INTR_FLAG_LEVEL and DDI_INTR_FLAG_EDGE flags.

ddi_intr_add_handler(9F)

Adds an interrupt handler.

ddi_intr_dup_handler(9F)

Use with MSI-X only. Copies an address and data pair for an allocated interrupt vector to an unused interrupt vector on the same device.

ddi_intr_remove_handler(9F)

Removes the specified interrupt handler.

ddi_intr_enable(9F)

Enables the specified interrupt.

ddi_intr_disable(9F)

Disables the specified interrupt.

ddi_intr_block_enable(9F)

Enables the specified range of interrupts. For MSI only.

ddi_intr_block_disable(9F)

Disables the specified range of interrupts. For MSI only.

ddi_intr_set_mask(9F)

Sets an interrupt mask if the specified interrupt is enabled.

ddi_intr_clr_mask(9F)

Clears an interrupt mask if the specified interrupt is enabled.

ddi_intr_get_pending(9F)

Reads the interrupt pending bit if one is supported by either the host bridge or the device.

Priority Management Functions

The following functions are used to obtain and set priority information:

ddi_intr_get_pri(9F)

Returns the current software priority setting for the specified interrupt.

ddi_intr_set_pri(9F)

Sets the interrupt priority level for the specified interrupt.

ddi_intr_get_hilevel_pri(9F)

Returns the minimum priority level for a high-level interrupt.

Soft Interrupt Functions

The following functions are used to manipulate soft interrupts and soft interrupt handlers:

ddi_intr_add_softint(9F)

Adds a soft interrupt handler.

ddi_intr_trigger_softint(9F)

Triggers the specified soft interrupt.

ddi_intr_remove_softint(9F)

Removes the specified soft interrupt handler.

ddi_intr_get_softint_pri(9F)

Returns the soft interrupt priority for the specified interrupt.

ddi_intr_set_softint_pri(9F)

Changes the relative soft interrupt priority for the specified soft interrupt.

New Interrupt Function Examples

This section provides examples for performing the following tasks:

• Changing soft interrupt priority

• Checking for pending interrupts

• Setting interrupt masks

• Clearing interrupt masks

---

Example 8–1 Using the ddi_inter_set_softint_pri() Function

/* Change the soft interrupt priority to 9 */ 
if (ddi_intr_set_softint_pri(mydev->mydev_softint_hdl, 9) !=
    DDI_SUCCESS) {
    cmn_err (CE_WARN, "ddi_intr_set_softint_pri failed");
}

---

---

Example 8–2 Using the ddi_intr_get_pending() Function

/* Check if an interrupt is pending */
if (ddi_intr_get_pending(mydevp->htable[0], &pending) != DDI_SUCCESS) {
    cmn_err(CE_WARN, "ddi_intr_get_pending() failed");
} else if (pending)
    cmn_err(CE_NOTE, "ddi_intr_get_pending(): Interrupt pending");

---

---

Example 8–3 Setting Interrupt Masks

Use the ddi_intr_set_mask(9F) function to set interrupt masking to prevent the device from receiving interrupts.

if ((ddi_intr_set_mask(mydevp->htable[0]) != DDI_SUCCESS))
    cmn_err(CE_WARN, "ddi_intr_set_mask() failed");

---

---

Example 8–4 Clearing Interrupt Masks

Use the ddi_intr_clr_mask(9F) function to clear interrupt masking. The ddi_intr_clr_mask(9F) function fails or if the specified interrupt is not enabled. If the ddi_intr_clr_mask(9F) function succeeds, the device starts generating interrupts.

if (ddi_intr_clr_mask(mydevp->htable[0]) != DDI_SUCCESS)
    cmn_err(CE_WARN, "ddi_intr_clr_mask() failed");

---

Legacy Interrupt Functions

To take advantage of the features of the new framework, developers need to use these new interfaces and avoid using the following interfaces, which are retained for compatibility purposes only.

Table 8–1 Legacy Interrupt Functions

Legacy Interrupt Functions 

Replacements 

ddi_add_intr(9F)

Three-step process: 

1. ddi_intr_alloc(9F)

2. ddi_intr_add_handler(9F)

3. ddi_intr_enable(9F)

ddi_add_softintr(9F)

ddi_intr_add_softint(9F)

ddi_dev_nintrs(9F)

ddi_intr_get_nintrs(9F)

ddi_get_iblock_cookie(9F)

Three-step process: 

1. ddi_intr_alloc(9F)

2. ddi_intr_get_pri(9F)

3. ddi_intr_free(9F)

ddi_get_soft_iblock_cookie(9F)

Three-step process: 

1. ddi_intr_add_softint(9F)

2. ddi_intr_get_softint_pri(9F)

3. ddi_intr_remove_softint(9F)

ddi_iblock_cookie(9S)

(void *) (uintptr_t )

ddi_idevice_cookie(9S)

Not applicable 

ddi_intr_hilevel(9F)

ddi_intr_get_hilevel_pri(9F)

ddi_remove_intr(9F)

ddi_intr_remove_handler(9F)

ddi_remove_softintr(9F)

ddi_intr_remove_softint(9F)

ddi_trigger_softintr(9F)

ddi_intr_trigger_softint(9F)

Registering Interrupts

Before a device driver can receive and service interrupts, the driver must register an interrupt handler with the system by calling ddi_add_intr(9F). Registering interrupts provides the system with a way to associate an interrupt handler with an interrupt specification. The interrupt handler is called when the device might have been responsible for the interrupt. The handler has the responsibility of determining whether it should handle the interrupt and, if so, of claiming that interrupt.

Registering Legacy Interrupts

To register a driver's interrupt handler, the driver typically performs the following steps in attach(9E).

1. Determine which types of interrupts are supported using ddi_intr_get_supported_types(9F).

2. Determine the number of supported interrupt types using ddi_intr_get_nintrs(9F).

3. Allocate memory for DDI interrupt handles using kmem_zalloc(9F).

4. For each interrupt type that you allocate, you must take the following steps:

   1. Get the priority for the interrupt using ddi_intr_get_pri(9F).

   2. If you need to set a new priority for the interrupt, use ddi_intr_set_pri(9F).

   3. Initialize the lock using mutex_init(9F).

   4. Register the handler for the interrupt using ddi_intr_add_handler(9F).

   5. Enable the interrupt using ddi_intr_enable(9F).

5. To free the interrupts, you must take the following steps for each interrupt:

   1. Disable each interrupt using ddi_intr_disable(9F).

   2. Remove the interrupt handler using ddi_intr_remove_handler(9F).

   3. Remove the lock using mutex_destroy(9F).

   4. Free the interrupt using ddi_intr_free(9F) and kmem_free(9F) to free memory that was allocated for DDI interrupt handles.

---

Example 8–5 Registering a Legacy Interrupt

The following example shows how to install an interrupt handler for a device called mydev. This example assumes that mydev supports one interrupt only.

/* Determine which types of interrupts supported */
ret = ddi_intr_get_supported_types(mydevp->mydev_dip, &type);

if ((ret != DDI_SUCCESS) || (!(type & DDI_INTR_TYPE_FIXED))) {
    cmn_err(CE_WARN, "Fixed type interrupt is not supported");
    return (DDI_FAILURE);
}

/* Determine number of supported interrupts */
ret = ddi_intr_get_nintrs(mydevp->mydev_dip, DDI_INTR_TYPE_FIXED, 
    &count);

/*
 * Fixed interrupts can only have one interrupt. Check to make
 * sure that number of supported interrupts and number of
 * available interrupts are both equal to 1.
 */
if ((ret != DDI_SUCCESS) || (count != 1)) {
    cmn_err(CE_WARN, "No fixed interrupts");
    return (DDI_FAILURE);
}

/* Allocate memory for DDI interrupt handles */
mydevp->mydev_htable = kmem_zalloc(sizeof (ddi_intr_handle_t), 
    KM_SLEEP);
ret = ddi_intr_alloc(mydevp->mydev_dip, mydevp->mydev_htable,
    DDI_INTR_TYPE_FIXED, 0, count, &actual, 0);

if ((ret != DDI_SUCCESS) || (actual != 1)) {
    cmn_err(CE_WARN, "ddi_intr_alloc() failed 0x%x", ret);
    kmem_free(mydevp->mydev_htable, sizeof (ddi_intr_handle_t));

    return (DDI_FAILURE);
}

/* Sanity check that count and available are the same. */
ASSERT(count == actual);

/* Get the priority of the interrupt */
if (ddi_intr_get_pri(mydevp->mydev_htable[0], &mydevp->mydev_intr_pri)) {
    cmn_err(CE_WARN, "ddi_intr_alloc() failed 0x%x", ret);

    (void) ddi_intr_free(mydevp->mydev_htable[0]);
    kmem_free(mydevp->mydev_htable, sizeof (ddi_intr_handle_t));

    return (DDI_FAILURE);
}

cmn_err(CE_NOTE, "Supported Interrupt priority = 0x%x", mydevp->mydev_intr_pri);

/* Test for high level mutex */
if (mydevp->mydev_intr_pri >= ddi_intr_get_hilevel_pri()) {
    cmn_err(CE_WARN, "Hi level interrupt not supported");

    (void) ddi_intr_free(mydevp->mydev_htable[0]);
    kmem_free(mydevp->mydev_htable, sizeof (ddi_intr_handle_t));

    return (DDI_FAILURE);
}

/* Initialize the mutex */
mutex_init(&mydevp->mydev_int_mutex, NULL, MUTEX_DRIVER,
    (void *)(uintptr_t)mydevp->mydev_intr_pri);

/* Register the interrupt handler */
if (ddi_intr_add_handler(mydevp->mydev_htable[0], mydev_intr, 
   (caddr_t)mydevp, NULL) !=DDI_SUCCESS) {
    cmn_err(CE_WARN, "ddi_intr_add_handler() failed");

    mutex_destroy(&mydevp->mydev_int_mutex);
    (void) ddi_intr_free(mydevp->mydev_htable[0]);
    kmem_free(mydevp->mydev_htable, sizeof (ddi_intr_handle_t));

    return (DDI_FAILURE);
}

/* Enable the interrupt */
if (ddi_intr_enable(mydevp->mydev_htable[0]) != DDI_SUCCESS) {
    cmn_err(CE_WARN, "ddi_intr_enable() failed");

    (void) ddi_intr_remove_handler(mydevp->mydev_htable[0]);
    mutex_destroy(&mydevp->mydev_int_mutex);
    (void) ddi_intr_free(mydevp->mydev_htable[0]);
    kmem_free(mydevp->mydev_htable, sizeof (ddi_intr_handle_t));

    return (DDI_FAILURE);
}
return (DDI_SUCCESS);
}

---

---

Example 8–6 Removing a Legacy Interrupt

The following example shows how legacy interrupts are removed.

/* disable interrupt */
(void) ddi_intr_disable(mydevp->mydev_htable[0]);

/* Remove interrupt handler */
(void) ddi_intr_remove_handler(mydevp->mydev_htable[0]);

/* free interrupt handle */
(void) ddi_intr_free(mydevp->mydev_htable[0]);

/* free memory */
kmem_free(mydevp->mydev_htable, sizeof (ddi_intr_handle_t));

---

Registering MSI Interrupts

To register a driver's interrupt handler, the driver usually performs the following steps in attach(9E).

1. Determine which types of interrupts are supported using ddi_intr_get_supported_types(9F).

2. Determine the number of supported MSI interrupt types using ddi_intr_get_nintrs(9F).

3. Allocate memory for the MSI interrupts using ddi_intr_alloc(9F).

4. For each interrupt type that you allocate, you must take the following steps:

   1. Get the priority for the interrupt using ddi_intr_get_pri(9F).

   2. If you need to set a new priority for the interrupt, use ddi_intr_set_pri().

   3. Initialize the lock using mutex_init(9F).

   4. Register the handler for the interrupt using ddi_intr_add_handler(9F).

5. Enable all the interrupts either as a block if ddi_intr_block_enable(9F) is supported or in a for() loop applying ddi_intr_enable(9F) to each interrupt individually.

---

Example 8–7 Registering a Set of MSI Interrupts

The following example illustrates how to register an MSI interrupt for a device called mydev.

/* Get supported interrupt types */
if (ddi_intr_get_supported_types(devinfo, &intr_types) != DDI_SUCCESS) {
    cmn_err(CE_WARN, "ddi_intr_get_supported_types failed");
    goto attach_fail;
}

if (intr_types & DDI_INTR_TYPE_MSI) 
    mydev_add_msi_intrs(mydevp);

/* Check count, available and actual interrupts */
static int
mydev_add_msi_intrs(mydev_t *mydevp)
{
    dev_info_t    *devinfo = mydevp->devinfo;
    int        count, avail, actual;
    int        x, y, rc, inum = 0;

    /* Get number of interrupts */
    rc = ddi_intr_get_nintrs(devinfo, DDI_INTR_TYPE_MSI, &count);
    if ((rc != DDI_SUCCESS) || (count == 0)) {
        cmn_err(CE_WARN, "ddi_intr_get_nintrs() failure, rc: %d, "
            "count: %d", rc, count);

        return (DDI_FAILURE);
    }

    /* Get number of available interrupts */
    rc = ddi_intr_get_navail(devinfo, DDI_INTR_TYPE_MSI, &avail);
    if ((rc != DDI_SUCCESS) || (avail == 0)) {
        cmn_err(CE_WARN, "ddi_intr_get_navail() failure, "
            "rc: %d, avail: %d/n", rc, avail);
        return (DDI_FAILURE);
    }
    if (avail < count) {
        cmn_err(CE_NOTE, "nitrs() returned %d, navail returned %d",
            count, avail);
    }

    /* Allocate memory for MSI interrupts */
    mydevp->intr_size = count * sizeof (ddi_intr_handle_t);
    mydevp->htable = kmem_alloc(mydevp->intr_size, KM_SLEEP);

    rc = ddi_intr_alloc(devinfo, mydevp->htable, DDI_INTR_TYPE_MSI, inum,
        count, &actual, DDI_INTR_ALLOC_NORMAL);

    if ((rc != DDI_SUCCESS) || (actual == 0)) {
        cmn_err(CE_WARN, "ddi_intr_alloc() failed: %d", rc);

        kmem_free(mydevp->htable, mydevp->intr_size);
        return (DDI_FAILURE);
    }

    if (actual < count) {
        cmn_err(CE_NOTE, "Requested: %d, Received: %d", count, actual);
    }

    mydevp->intr_cnt = actual;

    /*
     * Get priority for first msi, assume remaining are all the same
     */
    if (ddi_intr_get_pri(mydevp->htable[0], &mydev->intr_pri) !=
        DDI_SUCCESS) {
        cmn_err(CE_WARN, "ddi_intr_get_pri() failed");

        /* Free already allocated intr */
        for (y = 0; y < actual; y++) {
            (void) ddi_intr_free(mydevp->htable[y]);
        }

        kmem_free(mydevp->htable, mydevp->intr_size);
        return (DDI_FAILURE);
    }

    /* Call ddi_intr_add_handler() */
    for (x = 0; x < actual; x++) {
        if (ddi_intr_add_handler(mydevp->htable[x], mydev_intr,
           (caddr_t)mydevp, NULL) != DDI_SUCCESS) {
            cmn_err(CE_WARN, "ddi_intr_add_handler() failed");

            /* Free already allocated intr */
            for (y = 0; y < actual; y++) {
                (void) ddi_intr_free(mydevp->htable[y]);
            }

            kmem_free(mydevp->htable, mydevp->intr_size);
            return (DDI_FAILURE);
        }
    }

    (void) ddi_intr_get_cap(mydevp->htable[0], &mydevp->intr_cap);
    if (mydev->m_intr_cap & DDI_INTR_FLAG_BLOCK) {
        /* Call ddi_intr_block_enable() for MSI */
        (void) ddi_intr_block_enable(mydev->m_htable, mydev->m_intr_cnt);
    } else {
        /* Call ddi_intr_enable() for MSI non block enable */
        for (x = 0; x < mydev->m_intr_cnt; x++) {
            (void) ddi_intr_enable(mydev->m_htable[x]);
        }
    }

    return (DDI_SUCCESS);
}

---

---

Example 8–8 Removing MSI Interrupts

The following example shows how MSI interrupts are removed.

static void
mydev_rem_intrs(mydev_t *mydev)
{
    int        x;

    /* Disable all interrupts */
    if (mydev->m_intr_cap & DDI_INTR_FLAG_BLOCK) {
        /* Call ddi_intr_block_disable() */
        (void) ddi_intr_block_disable(mydev->m_htable, mydev->m_intr_cnt);
    } else {
        for (x = 0; x < mydev->m_intr_cnt; x++) {
            (void) ddi_intr_disable(mydev->m_htable[x]);
        }
    }

    /* Call ddi_intr_remove_handler() */
    for (x = 0; x < mydev->m_intr_cnt; x++) {
        (void) ddi_intr_remove_handler(mydev->m_htable[x]);
        (void) ddi_intr_free(mydev->m_htable[x]);
    }

    kmem_free(mydev->m_htable, mydev->m_intr_size);
}    

---

Interrupt Handler Responsibilities

The interrupt handler has a set of responsibilities to perform. Some responsibilities are required by the framework, and some responsibilities are required by the device. All interrupt handlers are required to do the following tasks:

• Determine whether the device is interrupting and possibly reject the interrupt.

  The interrupt handler must first examine the device to determine whether this device has issued the interrupt. If the device has not issued the interrupt, the handler must return DDI_INTR_UNCLAIMED. This step allows the implementation of device polling. Device polling tells the system whether this device, among a number of devices at the given interrupt priority level, has issued the interrupt.

• Inform the device that the device is being serviced.

  Informing a device about servicing is a device-specific operation that is required for the majority of devices. For example, SBus devices are required to interrupt until the driver tells the SBus devices to stop. This approach guarantees that all SBus devices that interrupt at the same priority level are serviced.

• Perform any I/O request-related processing.

  Devices interrupt for different reasons, such as transfer done or transfer error. This step can involve using data access functions to read the device's data buffer, examine the device's error register, and set the status field in a data structure accordingly. Interrupt dispatching and processing are relatively time consuming.

• Do any additional processing that could prevent another interrupt.

  For example, read the next item of data from the device.

• Return DDI_INTR_CLAIMED.

• MSI interrupts must always be claimed.

  Claiming an interrupt is optional for MSI-X interrupts. In either case, the ownership of the interrupt need not be checked, because MSI and MSI-X interrupts are not shared with other devices.

• Drivers that support hotplugging and multiple MSI or MSI-X interrupts should retain a separate interrupt for hotplug events and register a separate ISR (interrupt service routine) for that interrupt.

The following example shows an interrupt routine for a device called mydev.

---

Example 8–9 Interrupt Example

static uint_t
mydev_intr(caddr_t arg1, caddr_t arg2)
{
    struct mydevstate *xsp = (struct mydevstate *)arg1;
    uint8_t     status; 
    volatile  uint8_t  temp;

    /*
     * Claim or reject the interrupt.This example assumes
     * that the device's CSR includes this information.
     */
    mutex_enter(&xsp->high_mu);
    /* use data access routines to read status */
    status = ddi_get8(xsp->data_access_handle, &xsp->regp->csr);
    if (!(status & INTERRUPTING)) {
        mutex_exit(&xsp->high_mu);
        return (DDI_INTR_UNCLAIMED); /* dev not interrupting */
    }
    /*
     * Inform the device that it is being serviced, and re-enable
     * interrupts. The example assumes that writing to the
     * CSR accomplishes this. The driver must ensure that this data
     * access operation makes it to the device before the interrupt
     * service routine returns. For example, using the data access
     * functions to read the CSR, if it does not result in unwanted
     * effects, can ensure this.
     */
    ddi_put8(xsp->data_access_handle, &xsp->regp->csr,
        CLEAR_INTERRUPT | ENABLE_INTERRUPTS);
    /* flush store buffers */
    temp = ddi_get8(xsp->data_access_handle, &xsp->regp->csr);
    
    mutex_exit(&xsp->mu);
    return (DDI_INTR_CLAIMED);
}

---

Most of the steps performed by the interrupt routine depend on the specifics of the device itself. Consult the hardware manual for the device to determine the cause of the interrupt, detect error conditions, and access the device data registers.

Handling High-Level Interrupts

High-level interrupts are those interrupts that interrupt at the level of the scheduler and above. This level does not allow the scheduler to run. Therefore, high-level interrupt handlers cannot be preempted by the scheduler. High-level interrupts cannot rely on the scheduler, that is, they cannot block because of the scheduler. High-level interrupts can only use mutual exclusion locks for locking.

Because of this situation, the driver must use ddi_intr_hilevel(9F) to determine whether the driver is using high-level interrupts. If ddi_intr_hilevel(9F) returns true, the driver can fail to attach, or the driver can use a two-level scheme to handle interrupts.

The suggested method is to add a high-level interrupt handler, which simply triggers a lower-priority software interrupt to handle the device. The driver should allow more concurrency by using a separate mutex for protecting data from the high-level handler.

High-Level Mutexes

A mutex initialized with the interrupt block cookie that represents a high-level interrupt is known as ahigh-level mutex. While holding a high-level mutex, the driver is subject to the same restrictions as a high-level interrupt handler.

High-Level Interrupt Handling Example

In the following example, the high-level mutex (xsp->high_mu) is used only to protect data shared between the high-level interrupt handler and the soft interrupt handler. The protected data includes a queue used by both the high-level interrupt handler and the low-level handler, and a flag that indicates that the low-level handler is running. A separate low-level mutex (xsp->low_mu) protects the rest of the driver from the soft interrupt handler.

---

Example 8–10 Handling High-Level Interrupts With attach()

static int
mydevattach(dev_info_t *dip, ddi_attach_cmd_t cmd)
{
    struct mydevstate *xsp;
    [...]

    ret = ddi_intr_get_supported_types(dip, &type);
    if ((ret != DDI_SUCCESS) || (!(type & DDI_INTR_TYPE_FIXED))) {
        cmn_err(CE_WARN, "ddi_intr_get_supported_types() failed");
        return (DDI_FAILURE);
    }

    ret = ddi_intr_get_nintrs(dip, DDI_INTR_TYPE_FIXED, &count);

    /*
     * Fixed interrupts can only have one interrupt. Check to make
     * sure that number of supported interrupts and number of
     * available interrupts are both equal to 1.
     */
    if ((ret != DDI_SUCCESS) || (count != 1)) {
    cmn_err(CE_WARN, "No fixed interrupts found");
            return (DDI_FAILURE);
    }

    xsp->xs_htable = kmem_zalloc(count * sizeof (ddi_intr_handle_t),
        KM_SLEEP);

    ret = ddi_intr_alloc(dip, xsp->xs_htable, DDI_INTR_TYPE_FIXED, 0,
        count, &actual, 0);

    if ((ret != DDI_SUCCESS) || (actual != 1)) {
    cmn_err(CE_WARN, "ddi_intr_alloc failed 0x%x", ret");
        kmem_free(xsp->xs_htable, sizeof (ddi_intr_handle_t));
        return (DDI_FAILURE);
    }

    ret = ddi_intr_get_pri(xsp->xs_htable[0], &intr_pri);
    if (ret != DDI_SUCCESS) {
        cmn_err(CE_WARN, "ddi_intr_get_pri failed 0x%x", ret");
        (void) ddi_intr_free(xsp->xs_htable[0]);
        kmem_free(xsp->xs_htable, sizeof (ddi_intr_handle_t));
        return (DDI_FAILURE);
    }

    if (intr_pri >= ddi_intr_get_hilevel_pri()) {

    mutex_init(&xsp->high_mu, NULL, MUTEX_DRIVER,
        (void *) (uintptr_t)intr_pri);

    ret = ddi_intr_add_handler(xsp->xs_htable[0],
        mydevhigh_intr, (caddr_t)xsp, NULL);
    if (ret != DDI_SUCCESS) {
        cmn_err(CE_WARN, "ddi_intr_add_handler failed 0x%x", ret");
        mutex_destroy(&xsp>xs_int_mutex);
            (void) ddi_intr_free(xsp->xs_htable[0]);
            kmem_free(xsp->xs_htable, sizeof (ddi_intr_handle_t));
        return (DDI_FAILURE);
    }

    /* add soft interrupt */
    if (ddi_intr_add_softint(xsp->xs_dip, &xsp->xs_softint_hdl,
        DDI_INTR_SOFTPRI_MAX, xs_soft_intr, (caddr_t)xsp) !=
        DDI_SUCCESS) {
        cmn_err(CE_WARN, "add soft interrupt failed");
        mutex_destroy(&xsp->high_mu);
        (void) ddi_intr_remove_handler(xsp->xs_htable[0]);
            (void) ddi_intr_free(xsp->xs_htable[0]);
            kmem_free(xsp->xs_htable, sizeof (ddi_intr_handle_t));
        return (DDI_FAILURE);
    }


    xsp->low_iblock_cookie = DDI_INTR_SOFTPRI_MAX;

    mutex_init(&xsp->low_mu, NULL, MUTEX_DRIVER,
        (void *)xsp->low_iblock_cookie);

    } else {
    /*
     * regular interrupt registration continues from here
     * skip soft interrupt stuff
     */
    }

    return (DDI_SUCCESS);
}

---

The high-level interrupt routine services the device and queues the data. The high-level routine triggers a software interrupt if the low-level routine is not running, as the following example demonstrates.

---

Example 8–11 High-level Interrupt Routine

static uint_t
mydevhigh_intr(caddr_t arg1, caddr_t arg2)
{
   struct mydevstate    *xsp = (struct mydevstate *)arg1;
   uint8_t    status;
   volatile  uint8_t  temp;
   int    need_softint;

    mutex_enter(&xsp->high_mu);
    /* read status */
    status = ddi_get8(xsp->data_access_handle, &xsp->regp->csr);
    if (!(status & INTERRUPTING)) {
        mutex_exit(&xsp->high_mu);
        return (DDI_INTR_UNCLAIMED); /* dev not interrupting */
    }

    ddi_put8(xsp->data_access_handle,&xsp->regp->csr,
        CLEAR_INTERRUPT | ENABLE_INTERRUPTS);
    /* flush store buffers */
    temp = ddi_get8(xsp->data_access_handle, &xsp->regp->csr);

    read data from device and queue the data for the low-level interrupt handler;

    if (xsp->softint_running)
        need_softint = 0;
    else {
        xsp->softint_count++;
        need_softint = 1;
    }
    mutex_exit(&xsp->high_mu);

    /* read-only access to xsp->id, no mutex needed */
    if (need_softint) {
        ret = ddi_intr_trigger_softint(xsp->xs_softint_hdl, NULL);
        if (ret == DDI_EPENDING) {
            cmn_err(CE_WARN, "ddi_intr_trigger_softint() soft interrupt already "
                "pending for this handler");
        } else if (ret != DDI_SUCCESS) {
            cmn_err(CE_WARN, "ddi_intr_trigger_softint() failed");
        }           
    }

    return (DDI_INTR_CLAIMED);
}

---

The low-level interrupt routine is started by the high-level interrupt routine, which triggers a software interrupt. The low-level interrupt routine runs until there is nothing left to process, as the following example shows.

---

Example 8–12 Low-Level Soft Interrupt Routine

static uint_t
mydev_soft_intr(caddr_t arg1, caddr_t arg2)
{
    struct mydevstate *mydevp = (struct mydevstate *)arg1;
    [...]
    mutex_enter(&mydevp->low_mu);
    mutex_enter(&mydevp->high_mu);
    if (mydevp->softint_count > 1) {
        mydevp->softint_count--;
        mutex_exit(&mydevp->high_mu);
        mutex_exit(&mydevp->low_mu);
        return (DDI_INTR_CLAIMED);
    }

    if (queue empty) {
        mutex_exit(&mydevp->high_mu);
        mutex_exit(&mydevp->low_mu);
        return (DDI_INTR_UNCLAIMED);
    }

    mydevp->softint_running = 1;
    while (EMBEDDED COMMENT:data on queue) {
        ASSERT(mutex_owned(&mydevp->high_mu);
        dequeue data from high-level queue;
        mutex_exit(&mydevp->high_mu);
        normal interrupt processing
        mutex_enter(&mydevp->high_mu);
    }

    mydevp->softint_running = 0;
    mydevp->softint_count = 0;
    mutex_exit(&mydevp->high_mu);
    mutex_exit(&mydevp->low_mu);
    return (DDI_INTR_CLAIMED);
}