[peerdist] Allow for the use of a hosted cache server
Allow a PeerDist hosted cache server to be specified via the
${peerhost} setting, e.g.:
# Use 192.168.0.1 as hosted cache server
set peerhost 192.168.0.1
Note that this simply treats the hosted cache server as a permanently
discovered peer for all segments.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[peerdist] Allow PeerDist to be globally enabled or disabled
Allow the use of PeerDist content encoding to be enabled or disabled
via the ${peerdist} setting, e.g.:
# Disable PeerDist
set peerdist 0
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[tls] Add missing call to tls_tx_resume() when restarting negotiation
The restart of negotiation triggered by a HelloRequest currently does
not call tls_tx_resume() and so may end up leaving the connection in
an idle state in which the pending ClientHello is never sent.
Fix by calling tls_tx_resume() as part of tls_restart(), since the
call to tls_tx_resume() logically belongs alongside the code that sets
bits in tls->tx_pending.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[peerdist] Limit number of concurrent raw block downloads
Raw block downloads are expensive if the origin server uses HTTPS,
since each concurrent download will require local TLS resources
(including potentially large received encrypted data buffers).
Raw block downloads may also be prohibitively slow to initiate when
the origin server is using HTTPS and client certificates. Origin
servers for PeerDist downloads are likely to be running IIS, which has
a bug that breaks session resumption and requires each connection to
go through the full client certificate verification.
Limit the total number of concurrent raw block downloads to ameliorate
these problems.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[peerdist] Start block download timers from within opener methods
Move the responsibility for starting the block download timers from
peerblk_expired() to peerblk_raw_open() and peerblk_retrieval_open(),
in preparation for adding the ability to defer calls to
peerblk_raw_open() via a block download queue.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[vlan] Provide vlan_netdev_rx() and vlan_netdev_rx_err()
The Hermon driver uses vlan_find() to identify the appropriate VLAN
device for packets that are received with the VLAN tag already
stripped out by the hardware. Generalise this capability and expose
it for use by other network card drivers.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[tcp] Display "connecting" status until connection is established
Provide increased visibility into the progress of TCP connections by
displaying an explicit "connecting" status message while waiting for
the TCP handshake to complete.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[tls] Display cross-certificate and OCSP status messages
TLS connections will almost always create background connections to
perform cross-signed certificate downloads and OCSP checks. There is
currently no direct visibility into which checks are taking place,
which makes troubleshooting difficult in the absence of either a
packet capture or a debug build.
Use the job progress message buffer to report the current cross-signed
certificate download or OCSP status check, where applicable.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Record the session ID (if any) provided by the server and attempt to
reuse it for any concurrent connections to the same server.
If multiple connections are initiated concurrently (e.g. when using
PeerDist) then defer sending the ClientHello for all but the first
connection, to allow time for the first connection to potentially
obtain a session ID (and thereby speed up the negotiation for all
remaining connections).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[ethernet] Use standard 1500 byte MTU unless explicitly overridden
Devices that support jumbo frames will currently default to the
largest possible MTU. This assumption is valid for virtual adapters
such as virtio-net, where the MTU must have been configured by a
system administrator, but is unsafe in the general case of a physical
adapter.
Default to the standard Ethernet MTU, unless explicitly overridden
either by the driver or via the ${netX/mtu} setting.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[rndis] Clean up error handling path in register_rndis()
Avoid calling rndis_halt() and rndis->op->close() twice if the call to
register_netdev() fails.
Reported-by: Roman Kagan <rkagan@virtuozzo.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
register_netdev expects ->hw_addr and ->ll_addr to be already filled,
so move it towards the end of register_rndis, after the respective
fields have been successfully queried from the underlying device.
Signed-off-by: Roman Kagan <rkagan@virtuozzo.com>
Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[http] Work around stateful authentication schemes
As pointedly documented in RFC7230 section 2.3, HTTP is a stateless
protocol: each request message can be understood in isolation from any
other requests or responses. Various authentication schemes such as
NTLM break this fundamental property of HTTP and rely on the same TCP
connection being reused.
Work around these broken authentication schemes by ensuring that the
most recently pooled connection is reused for the subsequent
authentication retry.
Reported-by: Andreas Hammarskjöld <junior@2PintSoftware.com>
Tested-by: Andreas Hammarskjöld <junior@2PintSoftware.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[tls] Ensure that window change is propagated to plainstream interface
The cipherstream xfer_window_changed() message is used to retrigger
the TLS transmit state machine. If the transmit state machine is
idle, then the window change message will not be propagated to the
plainstream interface. This can potentially cause the plainstream
interface peer (e.g. httpcore) to block waiting for a window change
message that will never arrive.
Fix by ensuring that the window change message is propagated to the
plainstream interface if the transmit state machine is idle. (If the
transmit state machine is not idle then the plainstream window will be
zero anyway.)
Signed-off-by: Michael Brown <mcb30@ipxe.org>
In TLS terminology a session conceptually spans multiple individual
connections, and essentially represents the stored cryptographic state
(master secret and cipher suite) required to establish communication
without going through the certificate and key exchange handshakes.
Rename tls_session to tls_connection in order to make the name
tls_session available to represent the session state.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[tls] Ensure received data list is initialised before calling tls_free()
A failure in tls_generate_random() will result in a call to ref_put()
before the received data list has been initialised, which will cause
free_tls() to attempt to traverse an uninitialised list.
Fix by ensuring that all fields referenced by free_tls() are
initialised before any of the potential failure paths.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[lacp] Check the partner's own state when checking for blocked links
The blocked link test in eth_slow_lacp_rx() is performed before the
actor TLV is copied to the partner TLV, and so must test the actor
state field rather than the partner state field.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[lacp] Mark link as blocked if partner is not yet up and running
Mark the link as blocked if the LACP partner is not reporting itself
as being in sync, collecting, and distributing.
This matches the behaviour for STP: we mark the link as blocked if we
detect that the switch is actively blocking traffic, in order to
extend the DHCP discovery period and so prevent boot failures on
switches that take an excessively long time to enable ports.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The iSCSI root path may contain a literal IPv6 address. Update the
parser to handle this address format correctly.
Signed-off-by: Hannes Reinecke <hare@suse.de>
Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[netdevice] Make netdev_irq_enabled() independent of netdev_irq_supported()
The UNDI layer uses the NETDEV_IRQ_ENABLED flag to choose whether to
return PXENV_UNDI_ISR_OUT_OURS or PXENV_UNDI_ISR_OUT_NOT_OURS for a
given interrupt. For a network device that does not support
interrupts, the flag will never be set and so pxenv_undi_isr() will
always return PXENV_UNDI_ISR_OUT_NOT_OURS. This causes some NBPs
(such as lpxelinux.0) to hang.
Redefine NETDEV_IRQ_ENABLED as a simple administrative flag which can
be set even on network devices that do not support interrupts. This
allows pxenv_undi_isr() (which is the sole user of NETDEV_IRQ_ENABLED)
to function as expected by lpxelinux.0.
Signed-off-by: Martin Habets <mhabets@solarflare.com>
Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[http] Report unsuccessful response status lines at DBGVL_LOG
The precise HTTP response status code is currently visible only at
DBGLVL_EXTRA. Allow for easier debugging by reporting the whole
status line at DBGLVL_LOG for any unsuccessful responses.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[http] Handle parsing of WWW-Authenticate header within authentication scheme
Allow individual authentication schemes to parse WWW-Authenticate
headers that do not comply with RFC2617.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[http] Gracefully handle offers of multiple authentication schemes
Servers may provide multiple WWW-Authenticate headers, each offering a
different authentication scheme. We currently fail the request as
soon as we encounter an unrecognised scheme, which prevents subsequent
offers from succeeding.
Fix by silently ignoring headers for schemes that we do not recognise.
If no schemes are recognised then the request will eventually fail
anyway due to the 401 response code.
If multiple schemes are supported, arbitrarily choose the scheme
appearing first within the response headers.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[crypto] Fail fast if cross-certificate source is empty
In fully self-contained deployments it may be desirable to build iPXE
with an empty CROSSCERT source to avoid talking to external services.
Add an explicit check for this case and make validator_start_download
fail immediately if the base URI is empty.
Signed-off-by: Ladi Prosek <lprosek@redhat.com>
Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Expose the underlying hardware address as a setting. For IPoIB
devices, this provides scripts with access to the Infiniband GUID.
Requested-by: Allen, Benjamin S. <bsallen@alcf.anl.gov>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[peerdist] Gather and report peer statistics during download
Record and report the number of peers (calculated as the maximum
number of peers discovered for a block's segment at the time that the
block download is complete), and the percentage of blocks retrieved
from peers rather than from the origin server.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[netdevice] Cancel all pending transmissions on any transmit error
Some external code (such as the UEFI UNDI driver for the Realtek USB
NIC on a Microsoft Surface Book) will block during transmission
attempts and can take several seconds to report a transmit error. If
there is a large queue of pending transmissions, then the accumulated
time from a series of such failures can easily exceed the EFI watchdog
timeout, resulting in what appears to be a system lockup followed by a
reboot.
Work around this problem by immediately cancelling any pending
transmissions as soon as any transmit error occurs.
The only expected transmit error under normal operation is ENOBUFS
arising when the hardware transmit queue is full. By definition, this
can happen only for drivers that do not utilise deferred
transmissions, and so this new behaviour will not affect these
drivers.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Support renegotiation with servers supporting RFC5746. This allows
for the use of per-directory client certificates.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[tls] Keep cipherstream window open until TLS negotiation is complete
When performing a SAN boot, the plainstream window size will be zero
(since this is the mechanism used internally to indicate that no data
should be fetched via the initial request). This zero value currently
propagates to the advertised TCP window size, which prevents the TLS
negotiation from completing.
Fix by ensuring that the cipherstream window is held open until TLS
negotiation is complete, and only then falling back to passing through
the plainstream window size.
Reported-by: John Wigley <johnwigley#ipxe@acorna.co.uk>
Tested-by: John Wigley <johnwigley#ipxe@acorna.co.uk>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
As of kernel 4.11, the LIO target will propose a value for
FirstBurstLength if the initiator did not do so. This is entirely
redundant in our case, since FirstBurstLength is defined by RFC 3720
to be
"Irrelevant when: ( InitialR2T=Yes and ImmediateData=No )"
and we already enforce both InitialR2T=Yes and ImmediateData=No in our
initial proposal. However, LIO (arguably correctly) complains when we
do not respond to its redundant proposal of an already-irrelevant
value.
Fix by always proposing the default value for FirstBurstLength.
Debugged-by: Patrick Seeburger <info@8bit.de>
Tested-by: Patrick Seeburger <info@8bit.de>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[http] Notify data transfer interface when underlying connection is ready
HTTP implements xfer_window_changed() on the underlying server
connection using http_step(), which does not propagate the window
change notification to the data transfer interface. This breaks the
multipath-capable SAN boot code, which relies on the window change
notification to discover that the HTTP block device is ready for
commands to be issued.
Fix by sending xfer_window_changed() in http_step() once the
underlying connection has been determined to be ready.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Describe all SAN devices via ACPI tables such as the iBFT. For tables
that can describe only a single device (i.e. the aBFT and sBFT), one
table is installed per device. For multi-device tables (i.e. the
iBFT), all devices are described in a single table.
An underlying SAN device connection may be closed at the time that we
need to construct an ACPI table. We therefore introduce the concept
of an "ACPI descriptor" which enables the SAN boot code to maintain an
opaque pointer to the underlying object, and an "ACPI model" which can
build tables from a list of such descriptors. This separates the
lifecycles of ACPI descriptions from the lifecycles of the block
device interfaces, and allows for construction of the ACPI tables even
if the block device interface has been closed.
For a multipath SAN device, iPXE will wait until sufficient
information is available to describe all devices but will not wait for
all paths to connect successfully. For example: with a multipath
iSCSI boot iPXE will wait until at least one path has become available
and name resolution has completed on all other paths. We do this
since the iBFT has to include IP addresses rather than DNS names. We
will commence booting without waiting for the inactive paths to either
become available or close; this avoids unnecessary boot delays.
Note that the Linux kernel will refuse to accept an iBFT with more
than two NIC or target structures. We therefore describe only the
NICs that are actually required in order to reach the described
targets. Any iBFT with at most two targets is therefore guaranteed to
describe at most two NICs.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[infiniband] Return status code from ib_create_cq() and ib_create_qp()
Any underlying errors arising during ib_create_cq() or ib_create_qp()
are lost since the functions simply return NULL on error. This makes
debugging harder, since a debug-enabled build is required to discover
the root cause of the error.
Fix by returning a status code from these functions, thereby allowing
any underlying errors to be propagated.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Michael Brown