Passthrough a device described in the Device Tree to a guest
============================================================

The example will use the secondary network card for the midway server.

1) Mark the device to let Xen know the device will be used for passthrough.
This is done in the device tree node describing the device by adding the
property "xen,passthrough". The command to do it in U-Boot is:

    fdt set /soc/ethernet@fff51000 xen,passthrough

2) Create a partial device tree describing the device. The IRQ are mapped
1:1 to the guest (i.e VIRQ == IRQ). For MMIO, you will have to find a hole
in the guest memory layout (see xen/include/public/arch-arm.h, note that
the layout is not stable and can change between versions of Xen). Please
be aware that passing a partial device tree to a VM is a powerful tool,
use it with care. In production, only allow assignment of devices which
have been previously tested and known to work correctly when given to
guests.

/dts-v1/;

/ {
    /* #*cells are here to keep DTC happy */
    #address-cells = <2>;
    #size-cells = <2>;

    aliases {
        net = &mac0;
    };

    passthrough {
        compatible = "simple-bus";
        ranges;
        #address-cells = <2>;
        #size-cells = <2>;
        mac0: ethernet@10000000 {
            compatible = "calxeda,hb-xgmac";
            reg = <0 0x10000000 0 0x1000>;
            interrupts = <0 80 4  0 81 4  0 82 4>;
        };
    };
};

Note:
    * The interrupt-parent property will be added by the toolstack in the
    root node;
    * The following properties are mandatory with the /passthrough node:
        - compatible: It should always contain "simple-bus"
        - ranges
        - #address-cells
        - #size-cells
    * See http://www.devicetree.org/Device_Tree_Usage for more
    information about device tree.
    * In this example, the device MMIO region is placed at a different
    address (0x10000000) compared to the host address (0xfff51000)

3) Compile the partial guest device with dtc (Device Tree Compiler).
For our purpose, the compiled file will be called guest-midway.dtb and
placed in /root in DOM0.

3) Add the following options in the guest configuration file:

device_tree = "/root/guest-midway.dtb"
dtdev = [ "/soc/ethernet@fff51000" ]
irqs = [ 112, 113, 114 ]
iomem = [ "0xfff51,1@0x10000" ]

Please refer to your platform docs for the MMIO ranges and interrupts.

They can also be calculated from the original device tree (not
recommended). You can read about the "interrupts" property format in the
device tree bindings of the interrupt controller of your platform. For
example, in the case of GICv2 see [arm,gic.yaml]; in the case of GICv3
see [arm,gic-v3.yaml] in the Linux repository. For both GICv2 and GICv3
the "interrupts" property format is the same: the first cell is the
interrupt type, and the second cell is the interrupt number.  Given that
SPI numbers start from 32, in this example 80 + 32 = 112.

See man [xl.cfg] for the iomem format. The reg property is just a pair
of address, then size numbers, each of them can occupy 1 or 2 cells.


Dom0-less Device Passthrough
============================

The partial device tree for dom0-less guests should have the following
properties for each node corresponding to a physical device to assign to
the guest:

- xen,reg

  The xen,reg property is an array of:

    <phys_addr size guest_addr>

  They specify the physical address and size of the device memory
  ranges together with the corresponding guest address to map them to.
  The size of `phys_addr' and `guest_addr' is determined by
  #address-cells, the size of `size' is determined by #size-cells, of
  the partial device tree.
  The memory will be mapped as device memory in the guest (Device-nGnRE).

- xen,path

  A string property representing the path in the host device tree to the
  corresponding device node.

- xen,force-assign-without-iommu

  If xen,force-assign-without-iommu is present, Xen allows to assign a
  device even if it is not behind an IOMMU. This renders your platform
  *unsafe* if the device is DMA-capable.

In addition, a special /gic node is expected as a placeholder for the
full GIC node that will be added by Xen for the guest. /gic can be
referenced by other properties in the device tree fragment. For
instance, it can be referenced by interrupt-parent under a device node.
Xen will take care of replacing the "gic" placeholder node for a
complete GIC node while retaining all the references correctly. The new
GIC node created by Xen is a regular interrupt-controller@<unit> node.

    gic: gic {
        #interrupt-cells = <0x3>;
        interrupt-controller;
    };

Note that the #interrupt-cells and interrupt-controller properties are
not actually required, however, DTC expects them to be present if gic is
referenced by interrupt-parent or similar.


Example
=======

The following is a real-world example of a device tree fragment to
assign a network card to a dom0-less guest on Xilinx Ultrascale+ MPSoC:

/dts-v1/;

/ {
    #address-cells = <2>;
    #size-cells = <1>;

    gic: gic {
        #interrupt-cells = <3>;
        interrupt-controller;
    };

    passthrough {
        compatible = "simple-bus";
        ranges;
        #address-cells = <2>;
        #size-cells = <1>;

        misc_clk: misc_clk {
            #clock-cells = <0>;
            clock-frequency = <0x7735940>;
            compatible = "fixed-clock";
        };

        ethernet@ff0e0000 {
            compatible = "cdns,zynqmp-gem";
            status = "okay";
            reg = <0x0 0xff0e0000 0x1000>;
            clock-names = "pclk", "hclk", "tx_clk", "rx_clk";
            #address-cells = <1>;
            #size-cells = <0>;
            clocks = <&misc_clk &misc_clk &misc_clk &misc_clk>;
            phy-mode = "rgmii-id";
            xlnx,ptp-enet-clock = <0>;
            local-mac-address = [00 0a 35 00 22 01];
            interrupt-parent = <&gic>;
            interrupts = <0 63 0x4 0 63 0x4>;
            xen,path = "/amba/ethernet@ff0e0000";
            xen,reg = <0x0 0xff0e0000 0x1000 0x0 0xff0e0000>;

            phy@c {
                reg = <0xc>;
                ti,rx-internal-delay = <0x8>;
                ti,tx-internal-delay = <0xa>;
                ti,fifo-depth = <0x1>;
                ti,rxctrl-strap-worka;
            };
        };
    };
};


[arm,gic.yaml]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/interrupt-controller/arm,gic.yaml
[arm,gic-v3.yaml]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/devicetree/bindings/interrupt-controller/arm,gic-v3.yaml
[xl.cfg]: https://xenbits.xen.org/docs/unstable/man/xl.cfg.5.html
