ARM SMMUv3 architecture in linux

ARM SMMUv3 architecture in linux

-v0.1 2017.3.12 Sherlock init
-v0.2 2017.4.17 Sherlock add usage part

In [1], we know how SMMU driver has been loaded in ACPI.

During the driver loading, arm_smmu_device_probe will be called. Most part of
this function is to initiate SMMU hardware.

Then add smmu to a global list by: iommu_device_register(&smmu->iommu)

Then bus_set_iommu(&pci_bus_type, &arm_smmu_ops), set iommu_ops for a specific
bus, and call iommu_bus_init(bus, ops) -> bus_register_notifier(bus, nb) to
register a notifier_block to above bus. the notifier callback is iommu_bus_notifier
which will call add_device in SMMU’s iommu_ops:

/* drivers/iommu/arm-smmu-v3.c */arm_smmu_device_probe    --> bus_set_iommu(&pci_bus_type, &arm_smmu_ops)        --> iommu_bus_init(bus, ops)                /* here notifier's callback is iommu_bus_notifier */            --> bus_register_notifier(bus, nb)/* drivers/iommu/iommu.c */iommu_bus_notifier        /* action == BUS_NOTIFY_ADD_DEVICE, here ops is iommu_ops */    --> ops->add_device

When a device is adding to a bus, above notifier callback will be called:

/* drivers/base/core.c */device_add(struct device *dev)    --> blocking_notifier_call_chain(&dev->bus->p->bus_notifier,                                     BUS_NOTIFY_ADD_DEVICE, dev);

/* In drivers/iommu/arm-smmu-v3.c, create related iommu_group for dev */arm_smmu_add_device(struct device *dev)        /* reture an iommu_group about this device */    --> iommu_group_get_for_dev(dev)        --> ops->device_group(dev)                /* there is a algrithm to find/create one iommu_group?? */            --> pci_device_group            /* arm_smmu_domain_alloc, one iommu_group <-> one iommu_domain */        --> __iommu_domain_alloc            /* add a dev to a iommu_group */        --> iommu_group_add_device(group, dev)            --> __iommu_attach_device                domain->ops->attach_dev(domain, dev)                    --> arm_smmu_domain_finalise(domain)                        /* important!!! */                    --> arm_smmu_install_ste_for_dev(dev->iommu_fwspec)                /* here is another notifier call chain call? */            --> blocking_notifier_call_chain(&group->notifier,                IOMMU_GROUP_NOTIFY_ADD_DEVICE, dev);

The usage of smmu is: 1. DMA, 2. VM

• DMA usage

  We can go on in [1] to see how a PCIe device use SMMU to do DMA

  pci_device_add          /* In v4.12, this function will be moved to really_probe */      --> pci_dma_configure          --> acpi_dma_configure              --> arch_setup_dma_ops                  --> __iommu_setup_dma_ops                      --> do_iommu_attach                              /*                               * above __iommu_domain_alloc just allocate an                               * iommu_domain. here we initialize the iova domain                               * in it.                               * we do not do the map here.                               */                          --> iommu_dma_init_domain(domain, dma_base, size, dev)                          --> dev->dma_ops = &iommu_dma_ops

• VM usage

  to do…

Reference

[1] http://blog.csdn.net/scarecrow_byr/article/details/53844162