The Arrakis backport to Barrelfish.
authorSimon Peter <simpeter@cs.washington.edu>
Fri, 28 Feb 2014 08:54:25 +0000 (09:54 +0100)
committerGerd Zellweger <mail@gerdzellweger.com>
Fri, 28 Feb 2014 08:54:25 +0000 (09:54 +0100)
commitde906f6982f0c737093203d0ecdc34e1b9063a25
treeea5a1e3908bb03ff7cefd86ab1d8826e3e2bddae
parentd04b2e0c9aa85cf271a08903f50b93e90de738df
The Arrakis backport to Barrelfish.

Here are the macro changes:
* Made lwIP an optional network stack
  * Moved lwIP headers out of the general include directory
  * Parallel hake
  * CPU-virtualized ("arrakis") applications
  * libarrakis is a new libOS that uses CPU features directly
  * arrakismon is a new monitor for Arrakis apps
  * QEMU Multiboot support instead of GRUB (enables KVM)
  * e10k virtual function driver
  * Software virtual->physical mapping (substitutes IOMMU)
  * To be replaced soon with an Intel IOMMU driver
  * Better GDB and stack tracing support
  * New POSIX support
     * pthreads, inet_*(), usleep(), syslimits.h, sigaction(), mman.h,
     * resource.h, sendmsg(), recvmsg(), mkfifo(), syslog.h, sysexits.h,
     * fcntl(), getegid(), getgid(), pipe(), rlimits
     * epoll support
     * Separate waitset for interrupts
  * Thread TLS key and ID support
  * Moved PCRE headers to include/, where they usually are
  * Intel performance counters support
  * Handle boot-time RAM overflow (for machines with >=64GB RAM)
  * Arranet network stack
  * Fix for build of lib/barrelfish/arch/x86_64/dispatch.c without -O2
  * New comandline options to pci daemon:
     * maxvfs=x, x is number of virtual functions to create for each
     * device supporting it
     * skb_bridge_program=x, x is SKB bridge programming script to use
     * bridge_bios is a script that uses the BIOS pre-set values, like Linux
   * Correctly build machine-optimized newlib parts
   * Library versions of e1000 and e10k drivers
   * Support for virtualization in e10k driver
   * e10k datacenter control plane daemon
   * Modified hellotest to be both a Barrelfish and an Arrakis application

I've decided not to include the lwIP 1.4.1 code, as the port is
not finished and who wants to use lwIP anyway with the Arrakis
network stack and mTCP being released soon. If you want this,
let me know (it's hugely superior to lwIP 1.3.1).

I've made the code backwards-compatible with Barrelfish. The
default should do what Barrelfish does. One can specify
commandline options to the PCI daemon and e10k driver to
configure I/O virtualization and the optional arrakismon allows
loading CPU-virtualized "arrakis" programs.

There are more pending changes, such as support for hardware
rate limiting and weighted round robin packet scheduling, but
I'd like these to go in first, before I push the rest.
268 files changed:
devices/Hakefile
devices/e10k.dev
devices/e10k_vf.dev [new file with mode: 0644]
devices/pci_hdr0.dev
devices/pci_hdr1.dev
devices/pci_sr_iov_cap.dev [new file with mode: 0644]
hake/ArchDefaults.hs
hake/Config.hs.template
hake/Main.hs
hake/RuleDefs.hs
hake/hake.sh
hake/menu.lst.x86_64
hake/symbolic_targets.mk
if/Hakefile
if/arrakis.if [new file with mode: 0644]
if/e10k_vf.if [copied from include/lwip/barrelfish.h with 58% similarity]
include/Hakefile
include/arch/x86/barrelfish_kpi/perfmon.h
include/arch/x86_64/barrelfish/syscall_arch.h
include/arch/x86_64/barrelfish_kpi/eflags_arch.h
include/arpa/inet.h
include/arranet.h [new file with mode: 0644]
include/arranet_debug.h [new file with mode: 0644]
include/arranet_impl.h [new file with mode: 0644]
include/barrelfish/core_state.h
include/barrelfish/debug.h
include/barrelfish/deferred.h
include/barrelfish/domain.h
include/barrelfish/inthandler.h
include/barrelfish/memobj.h
include/barrelfish/spawn_client.h
include/barrelfish/sys_debug.h
include/barrelfish/threads.h
include/barrelfish/waitset_chan.h
include/barrelfish_kpi/sys_debug.h
include/contmng/netbench.h
include/fcntl.h
include/limits.h
include/netdb.h
include/netinet/in.h
include/netinet6/in6.h [new file with mode: 0644]
include/pcre.h [moved from lib/pcre/pcre.h with 100% similarity]
include/pcreposix.h [moved from include/pcre/pcreposix.h with 100% similarity]
include/pthread.h [new file with mode: 0644]
include/signal.h
include/spawndomain/spawndomain.h
include/sys/_iovec.h
include/sys/_pthreadtypes.h [new file with mode: 0644]
include/sys/_sockaddr_storage.h [new file with mode: 0644]
include/sys/_types.h
include/sys/epoll.h [new file with mode: 0644]
include/sys/mman.h [new file with mode: 0644]
include/sys/resource.h [new file with mode: 0644]
include/sys/signal.h [new file with mode: 0644]
include/sys/socket.h
include/sys/stat.h
include/sys/syslog.h [new file with mode: 0644]
include/sys/types.h
include/sysexits.h [new file with mode: 0644]
include/syslog.h [new file with mode: 0644]
include/unistd.h
include/vfs/fdtab.h
kernel/arch/x86/perfmon.c
kernel/arch/x86/perfmon_intel.c
kernel/arch/x86_64/irq.c
kernel/arch/x86_64/startup_arch.c
kernel/arch/x86_64/syscall.c
kernel/arch/x86_64/vmkit.c
kernel/dispatch.c
kernel/include/arch/x86/perfmon_intel.h
kernel/include/arch/x86_64/vmkit.h
lib/arranet/Hakefile [new file with mode: 0644]
lib/arranet/arranet.c [new file with mode: 0644]
lib/arranet/inet_chksum.c [new file with mode: 0644]
lib/arranet/inet_chksum.h [copied from include/ipv4/lwip/inet_chksum.h with 50% similarity]
lib/arranet/ip_addr.c [new file with mode: 0644]
lib/barrelfish/Hakefile
lib/barrelfish/arch/x86_64/dispatch.c
lib/barrelfish/arch/x86_64/ldt.c
lib/barrelfish/arch/x86_64/sys_debug.c
lib/barrelfish/debug.c
lib/barrelfish/deferred.c
lib/barrelfish/dispatch.c
lib/barrelfish/domain.c
lib/barrelfish/include/threads_priv.h
lib/barrelfish/include/waitset_chan_priv.h
lib/barrelfish/init.c
lib/barrelfish/inthandler.c
lib/barrelfish/spawn_client.c
lib/barrelfish/threads.c
lib/barrelfish/vspace/memobj_anon.c
lib/barrelfish/waitset.c
lib/lwip/Hakefile
lib/lwip/src/api/sockets.c
lib/lwip/src/include/arch/sys_arch.h [moved from include/arch/sys_arch.h with 100% similarity]
lib/lwip/src/include/ipv4/lwip/autoip.h [moved from include/ipv4/lwip/autoip.h with 100% similarity]
lib/lwip/src/include/ipv4/lwip/icmp.h [moved from include/ipv4/lwip/icmp.h with 100% similarity]
lib/lwip/src/include/ipv4/lwip/igmp.h [moved from include/ipv4/lwip/igmp.h with 100% similarity]
lib/lwip/src/include/ipv4/lwip/inet.h [moved from include/ipv4/lwip/inet.h with 100% similarity]
lib/lwip/src/include/ipv4/lwip/inet_chksum.h [moved from include/ipv4/lwip/inet_chksum.h with 100% similarity]
lib/lwip/src/include/ipv4/lwip/ip.h [moved from include/ipv4/lwip/ip.h with 100% similarity]
lib/lwip/src/include/ipv4/lwip/ip_addr.h [moved from include/ipv4/lwip/ip_addr.h with 91% similarity]
lib/lwip/src/include/ipv4/lwip/ip_frag.h [moved from include/ipv4/lwip/ip_frag.h with 100% similarity]
lib/lwip/src/include/ipv6/lwip/icmp.h [moved from include/ipv6/lwip/icmp.h with 100% similarity]
lib/lwip/src/include/ipv6/lwip/inet.h [moved from include/ipv6/lwip/inet.h with 100% similarity]
lib/lwip/src/include/ipv6/lwip/ip.h [moved from include/ipv6/lwip/ip.h with 100% similarity]
lib/lwip/src/include/ipv6/lwip/ip_addr.h [moved from include/ipv6/lwip/ip_addr.h with 100% similarity]
lib/lwip/src/include/lwip/api.h [moved from include/lwip/api.h with 100% similarity]
lib/lwip/src/include/lwip/api_msg.h [moved from include/lwip/api_msg.h with 100% similarity]
lib/lwip/src/include/lwip/arch.h [moved from include/lwip/arch.h with 100% similarity]
lib/lwip/src/include/lwip/barrelfish.h [copied from include/lwip/barrelfish.h with 100% similarity]
lib/lwip/src/include/lwip/debug.h [moved from include/lwip/debug.h with 100% similarity]
lib/lwip/src/include/lwip/def.h [moved from include/lwip/def.h with 100% similarity]
lib/lwip/src/include/lwip/dhcp.h [moved from include/lwip/dhcp.h with 100% similarity]
lib/lwip/src/include/lwip/dns.h [moved from include/lwip/dns.h with 100% similarity]
lib/lwip/src/include/lwip/err.h [moved from include/lwip/err.h with 100% similarity]
lib/lwip/src/include/lwip/init.h [moved from include/lwip/init.h with 100% similarity]
lib/lwip/src/include/lwip/mem.h [moved from include/lwip/mem.h with 100% similarity]
lib/lwip/src/include/lwip/memp.h [moved from include/lwip/memp.h with 100% similarity]
lib/lwip/src/include/lwip/memp_std.h [moved from include/lwip/memp_std.h with 100% similarity]
lib/lwip/src/include/lwip/netbuf.h [moved from include/lwip/netbuf.h with 100% similarity]
lib/lwip/src/include/lwip/netdb.h [moved from include/lwip/netdb.h with 100% similarity]
lib/lwip/src/include/lwip/netif.h [moved from include/lwip/netif.h with 100% similarity]
lib/lwip/src/include/lwip/netifapi.h [moved from include/lwip/netifapi.h with 100% similarity]
lib/lwip/src/include/lwip/opt.h [moved from include/lwip/opt.h with 100% similarity]
lib/lwip/src/include/lwip/pbuf.h [moved from include/lwip/pbuf.h with 100% similarity]
lib/lwip/src/include/lwip/raw.h [moved from include/lwip/raw.h with 100% similarity]
lib/lwip/src/include/lwip/sio.h [moved from include/lwip/sio.h with 100% similarity]
lib/lwip/src/include/lwip/snmp.h [moved from include/lwip/snmp.h with 100% similarity]
lib/lwip/src/include/lwip/snmp_asn1.h [moved from include/lwip/snmp_asn1.h with 100% similarity]
lib/lwip/src/include/lwip/snmp_msg.h [moved from include/lwip/snmp_msg.h with 100% similarity]
lib/lwip/src/include/lwip/snmp_structs.h [moved from include/lwip/snmp_structs.h with 100% similarity]
lib/lwip/src/include/lwip/sock_chan_support.h [moved from include/lwip/sock_chan_support.h with 96% similarity]
lib/lwip/src/include/lwip/sock_serialise.h [moved from include/lwip/sock_serialise.h with 100% similarity]
lib/lwip/src/include/lwip/sockets.h [moved from include/lwip/sockets.h with 96% similarity]
lib/lwip/src/include/lwip/stats.h [moved from include/lwip/stats.h with 100% similarity]
lib/lwip/src/include/lwip/sys.h [moved from include/lwip/sys.h with 100% similarity]
lib/lwip/src/include/lwip/tcp.h [moved from include/lwip/tcp.h with 100% similarity]
lib/lwip/src/include/lwip/tcpip.h [moved from include/lwip/tcpip.h with 100% similarity]
lib/lwip/src/include/lwip/udp.h [moved from include/lwip/udp.h with 100% similarity]
lib/lwip/src/include/lwipopts.h [moved from include/lwipopts.h with 100% similarity]
lib/lwip/src/include/netif/bfeth.h [moved from include/netif/bfeth.h with 100% similarity]
lib/lwip/src/include/netif/e1000.h [new file with mode: 0644]
lib/lwip/src/include/netif/etharp.h [moved from include/netif/etharp.h with 100% similarity]
lib/lwip/src/include/netif/loopif.h [moved from include/netif/loopif.h with 100% similarity]
lib/lwip/src/include/netif/slipif.h [moved from include/netif/slipif.h with 100% similarity]
lib/newlib/newlib/libc/Hakefile
lib/newlib/newlib/libc/machine/Hakefile [new file with mode: 0644]
lib/newlib/newlib/libc/sys/barrelfish/syscalls.c
lib/oldc/src/fileno.c
lib/posixcompat/Hakefile
lib/posixcompat/alarm.c
lib/posixcompat/close.c
lib/posixcompat/epoll.c [new file with mode: 0644]
lib/posixcompat/fcntl.c
lib/posixcompat/getegid.c [deleted file]
lib/posixcompat/geteuid.c
lib/posixcompat/gettimeofday.c
lib/posixcompat/inet_ntop.c [new file with mode: 0644]
lib/posixcompat/inet_pton.c [new file with mode: 0644]
lib/posixcompat/mkfifo.c [new file with mode: 0644]
lib/posixcompat/pipe.c
lib/posixcompat/pthreads.c [new file with mode: 0644]
lib/posixcompat/read.c
lib/posixcompat/readv.c
lib/posixcompat/select.c
lib/posixcompat/setrlimit.c [new file with mode: 0644]
lib/posixcompat/signal.c
lib/posixcompat/sleep.c [new file with mode: 0644]
lib/posixcompat/sockets.c
lib/posixcompat/unixsock.h
lib/posixcompat/wait.c
lib/posixcompat/write.c
lib/skb/skb_functions.c
lib/spawndomain/arch/x86/spawn_arch.c
lib/spawndomain/spawn.c
lib/vfs/Hakefile
lib/vfs/vfs_fd.c
tools/debug.gdb
tools/debug.sh
tools/elver/elver.c
tools/harness/machines/uw.py [new file with mode: 0644]
tools/harness/machines/uw_machinedata.py [new file with mode: 0644]
tools/harness/siteconfig/__init__.py
tools/harness/siteconfig/uw.py [new file with mode: 0644]
usr/acpi/acpi_service.c
usr/arrakismon/Hakefile [new file with mode: 0644]
usr/arrakismon/apic.c [new file with mode: 0644]
usr/arrakismon/apic.h [new file with mode: 0644]
usr/arrakismon/console.c [new file with mode: 0644]
usr/arrakismon/console.h [new file with mode: 0644]
usr/arrakismon/guest.c [new file with mode: 0644]
usr/arrakismon/guest.h [new file with mode: 0644]
usr/arrakismon/hdd.c [new file with mode: 0644]
usr/arrakismon/hdd.h [new file with mode: 0644]
usr/arrakismon/keyboard.c [new file with mode: 0644]
usr/arrakismon/keyboard.h [new file with mode: 0644]
usr/arrakismon/lpc.c [new file with mode: 0644]
usr/arrakismon/lpc.h [new file with mode: 0644]
usr/arrakismon/pc16550d.c [new file with mode: 0644]
usr/arrakismon/pc16550d.h [new file with mode: 0644]
usr/arrakismon/pci.c [new file with mode: 0644]
usr/arrakismon/pci.h [new file with mode: 0644]
usr/arrakismon/pci_devices.h [copied from include/lwip/barrelfish.h with 60% similarity]
usr/arrakismon/pci_host.c [new file with mode: 0644]
usr/arrakismon/pci_host.h [moved from include/lwip/barrelfish.h with 60% similarity]
usr/arrakismon/pci_hostbridge.c [new file with mode: 0644]
usr/arrakismon/ps.c [new file with mode: 0644]
usr/arrakismon/ps.h [new file with mode: 0644]
usr/arrakismon/realmode.c [new file with mode: 0644]
usr/arrakismon/realmode.h [new file with mode: 0644]
usr/arrakismon/svm.h [new file with mode: 0644]
usr/arrakismon/vmkitmon.c [new file with mode: 0644]
usr/arrakismon/vmkitmon.h [new file with mode: 0644]
usr/arrakismon/x86.h [new file with mode: 0644]
usr/bench/ahci/Hakefile
usr/bench/net_latency/elb/Hakefile
usr/bench/udp_echo/Hakefile [new file with mode: 0644]
usr/bench/udp_echo/Makefile [new file with mode: 0644]
usr/bench/udp_echo/run_multiloop.sh [new file with mode: 0644]
usr/bench/udp_echo/udp_echo.c [new file with mode: 0644]
usr/bench/udp_echo/udp_openloop.c [new file with mode: 0644]
usr/drivers/e1000/Hakefile
usr/drivers/e1000/e1000n.c
usr/drivers/e1000/e1000n_debug.c [deleted file]
usr/drivers/e10k/Hakefile
usr/drivers/e10k/e10k.h
usr/drivers/e10k/e10k_cdriver.c
usr/drivers/e10k/e10k_qdriver.c
usr/drivers/e10k/e10k_queue.h
usr/drivers/e10k/e10k_vf.c [new file with mode: 0644]
usr/drivers/e10k/sleep.h
usr/fish/Hakefile
usr/kaluga/Hakefile
usr/net-test/Hakefile
usr/openssh/config.h
usr/pci/Hakefile
usr/pci/pci.c
usr/pci/pci.h
usr/pci/pcie.c
usr/pci/pcimain.c
usr/ramfsd/main.c
usr/skb/Hakefile
usr/skb/eclipse_kernel/src/config.h
usr/skb/icparc_solver/eregex.c
usr/skb/programs/bridge_bios.pl [new file with mode: 0644]
usr/skb/programs/bridge_fake_bigfish.pl [new file with mode: 0644]
usr/slideshow/Hakefile
usr/spawnd/Hakefile
usr/startd/Hakefile
usr/startd/internal.h
usr/startd/main.c
usr/startd/spawn.c
usr/tests/e10ktest/Hakefile [new file with mode: 0644]
usr/tests/e10ktest/e10k_ctrl.c [new file with mode: 0644]
usr/tests/e10ktest/e10ktest.c [new file with mode: 0644]
usr/tests/e10ktest/e10ktest_latencies.c [new file with mode: 0644]
usr/tests/filetests/Hakefile
usr/tests/hellotest/Hakefile
usr/tests/hellotest/hellotest.c
usr/tests/net_tests/nfs_throughput/Hakefile
usr/tests/perfmontest/Hakefile
usr/tests/perfmontest/spin.c [new file with mode: 0644]
usr/tests/posix/Hakefile [new file with mode: 0644]
usr/tests/posix/socket_pipe.c [new file with mode: 0644]
usr/tests/testdesc/Hakefile
usr/tests/testdesc/testdesc-child.c
usr/vmkitmon/Hakefile