[http] Notify data transfer interface when underlying connection is ready
HTTP implements xfer_window_changed() on the underlying server
connection using http_step(), which does not propagate the window
change notification to the data transfer interface. This breaks the
multipath-capable SAN boot code, which relies on the window change
notification to discover that the HTTP block device is ready for
commands to be issued.
Fix by sending xfer_window_changed() in http_step() once the
underlying connection has been determined to be ready.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Describe all SAN devices via ACPI tables such as the iBFT. For tables
that can describe only a single device (i.e. the aBFT and sBFT), one
table is installed per device. For multi-device tables (i.e. the
iBFT), all devices are described in a single table.
An underlying SAN device connection may be closed at the time that we
need to construct an ACPI table. We therefore introduce the concept
of an "ACPI descriptor" which enables the SAN boot code to maintain an
opaque pointer to the underlying object, and an "ACPI model" which can
build tables from a list of such descriptors. This separates the
lifecycles of ACPI descriptions from the lifecycles of the block
device interfaces, and allows for construction of the ACPI tables even
if the block device interface has been closed.
For a multipath SAN device, iPXE will wait until sufficient
information is available to describe all devices but will not wait for
all paths to connect successfully. For example: with a multipath
iSCSI boot iPXE will wait until at least one path has become available
and name resolution has completed on all other paths. We do this
since the iBFT has to include IP addresses rather than DNS names. We
will commence booting without waiting for the inactive paths to either
become available or close; this avoids unnecessary boot delays.
Note that the Linux kernel will refuse to accept an iBFT with more
than two NIC or target structures. We therefore describe only the
NICs that are actually required in order to reach the described
targets. Any iBFT with at most two targets is therefore guaranteed to
describe at most two NICs.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some iSCSI targets send NOP-In. Rather than closing the connection
when we receive one, it is more user friendly to log a debug message
and keep the connection open. Eventually, it would be nice if iPXE
supported replying to NOP-Ins, but we might as well keep the
connection open until the target disconnects us.
Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[http] Cleanly shut down potentially looped interfaces
Use intfs_shutdown() and intfs_restart() to cleanly shut down multiple
interfaces that may loop back to the same object.
This fixes a regression introduced by commit daa8ed9 ("[interface]
Provide intf_reinit() to reinitialise nullified interfaces") which
broke the use of HTTP Basic and Digest authentication.
Reported-by: murmansk <murmansk@hotmail.com>
Reported-by: Brett Waldo <brettwaldo@gmail.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[cloud] Add ability to retrieve Google Compute Engine metadata
For some unspecified "security" reason, the Google Compute Engine
metadata server will refuse any requests that do not include the
non-standard HTTP header "Metadata-Flavor: Google".
Attempt to autodetect such requests (by comparing the hostname against
"metadata.google.internal"), and add the "Metadata-Flavor: Google"
header if applicable.
Enable this feature in the CONFIG=cloud build, and include a sample
embedded script allowing iPXE to boot from a script configured as
metadata via e.g.
# Create shared boot image
make bin/ipxe.usb CONFIG=cloud EMBED=config/cloud/gce.ipxe
# Configure per-instance boot script
gcloud compute instances add-metadata <instance> \
--metadata-from-file ipxeboot=boot.ipxe
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[iscsi] Avoid potential infinite loops during shutdown
The command and data interfaces may be connected to the same object.
Nullify the data interface before shutting down the control interface
to avoid potential infinite loops.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Some HTTP/2 servers send the header "Connection: upgrade, close". This
currently causes iPXE to fail due to the unrecognised "upgrade" token.
Fix by ignoring any unrecognised tokens in the "Connection" header.
Reported-by: Ján ONDREJ (SAL) <ondrejj@salstar.sk>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Resolve redirection URIs as being relative to the original HTTP
request URI, rather than treating them as being implicitly relative to
the current working URI.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Commit 09b057c ("[settings] Remove "uristring" setting type") removed
support for URI-encoded settings via the "uristring" setting type, on
the basis that such encoding was no longer necessary to avoid problems
with the command line parser.
Other valid use cases for the "uristring" setting type do exist: for
example, a password containing a '/' character expanded via
chain http://username:${password:uristring}@server.name/boot.php
Restore the existence of the "uristring" setting, avoiding the
potentially large stack allocations that were used in the old code
prior to commit 09b057c ("[settings] Remove "uristring" setting
type").
Requested-by: Robin Smidsrød <robin@smidsrod.no>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[http] Rewrite HTTP core to support content encodings
Rewrite the HTTP core to allow for the addition of arbitrary content
encoding mechanisms, such as PeerDist and gzip.
The core now exposes http_open() which can be used to create requests
with an explicitly selected HTTP method, an optional requested content
range, and an optional request body. A simple wrapper provides the
preexisting behaviour of creating either a GET request or an
application/x-www-form-urlencoded POST request (if the URI includes
parameters).
The HTTP SAN interface is now implemented using the generic block
device translator. Individual blocks are requested using http_open()
to create a range request.
Server connections are now managed via a connection pool; this allows
for multiple requests to the same server (e.g. for SAN blocks) to be
completely unaware of each other. Repeated HTTPS connections to the
same server can reuse a pooled connection, avoiding the per-connection
overhead of establishing a TLS session (which can take several seconds
if using a client certificate).
Support for HTTP SAN booting and for the Basic and Digest
authentication schemes is now optional and can be controlled via the
SANBOOT_PROTO_HTTP, HTTP_AUTH_BASIC, and HTTP_AUTH_DIGEST build
configuration options in config/general.h.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
iscsi_tx_done() is missing "break" statements at the end of each case.
(Fortunately, this happens not to cause a bug in practice, since
iscsi_login_request_done() is effectively a no-op when completing a
data-out PDU.)
Reported-by: Wissam Shoukair <wissams@mellanox.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[base16] Add buffer size parameter to base16_encode() and base16_decode()
The current API for Base16 (and Base64) encoding requires the caller
to always provide sufficient buffer space. This prevents the use of
the generic encoding/decoding functionality in some situations, such
as in formatting the hex setting types.
Implement a generic hex_encode() (based on the existing
format_hex_setting()), implement base16_encode() and base16_decode()
in terms of the more generic hex_encode() and hex_decode(), and update
all callers to provide the additional buffer length parameter.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Microsoft IIS supports only MD5-sess for Digest authentication.
Requested-by: Andreas Hammarskjöld <junior@2PintSoftware.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[legal] Relicense files under GPL2_OR_LATER_OR_UBDL
These files cannot be automatically relicensed by util/relicense.pl
since they either contain unusual but trivial contributions (such as
the addition of __nonnull function attributes), or contain lines
dating back to the initial git revision (and so require manual
knowledge of the code's origin).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[build] Expose build timestamp, build name, and product names
Expose the build timestamp (measured in seconds since the Epoch) and
the build name (e.g. "rtl8139.rom" or "ipxe.efi"), and provide the
product name and product short name in a single centralised location.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Parse the sense data to extract the reponse code, the sense key, the
additional sense code, and the additional sense code qualifier.
Originally-implemented-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[http] Accept Content-Length header with trailing whitespace
At least one HTTP server (Google's OCSP responder) has been observed
to generate a Content-Length header with trailing whitespace.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[http] Automatically retry request on a 503 Service Unavailable
A web server may return a 503 Service Unavailable response along with
a Retry-After header to direct the client to retry the request at a
later time.
The Retry-After header may be a number of seconds, or a full HTTP
timestamp (e.g. "Fri, 7 Mar 2014 17:22:14 GMT"). We have no
reasonable way of parsing a full HTTP timestamp; if the server chooses
to use this format then we simply retry after a fixed 5-second delay.
As per RFC 2616, in the absence of a Retry-After header we treat a
status code of 503 Service Unavailable as being equivalent to 500
Internal Server Error, and immediately fail the request.
Requested-by: Suresh Sundriyal <ssundriy@vmware.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Add support for parsing of URIs containing literal IPv6 addresses
(e.g. "http://[fe80::69ff:fe50:5845%25net0]/boot.ipxe").
Duplicate URIs by directly copying the relevant fields, rather than by
formatting and reparsing a URI string. This relaxes the requirements
on the URI formatting code and allows it to focus on generating
human-readable URIs (e.g. by not escaping ':' characters within
literal IPv6 addresses). As a side-effect, this allows relative URIs
containing parameter lists (e.g. "../boot.php##params") to function
as expected.
Add validity check for FTP paths to ensure that only printable
characters are accepted (since FTP is a human-readable line-based
protocol with no support for character escaping).
Construct TFTP next-server+filename URIs directly, rather than parsing
a constructed "tftp://..." string,
Add self-tests for URI functions.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Note that IANA has not yet assigned a DHCPv6 option code for the
syslog server. When a code is assigned, the definition of
DHCPV6_LOG_SERVERS should be updated. Until then, an IPv6 address of
a syslog server can be configured manually using e.g.
set syslog6 3ffe:302:11:2::8309
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[settings] Explicitly separate the concept of a completed fetched setting
The fetch_setting() family of functions may currently modify the
definition of the specified setting (e.g. to add missing type
information). Clean up this interface by requiring callers to provide
an explicit buffer to contain the completed definition of the fetched
setting, if required.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Allow HTTP POST requests to be generated when the URI includes a
parameter list. For example:
#!ipxe
params
param mac ${net0/mac}
param uuid ${uuid}
param asset ${asset}
chain http://boot.ipxe.org/demo/boot.php##params
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Commit 501527d ("[http] Treat any unexpected connection close as an
error") introduced a regression causing HTTP SAN booting to fail. At
the end of the response to the HEAD request, the call to http_done()
would erroneously believe that the server had disconnected in the
middle of the HTTP headers.
Fix by treating the header block from a HEAD request as a trailer
block. This fixes the problem and also simplifies the logic in
http_rx_header().
Reported-by: Shao Miller <shao.miller@yrdsb.edu.on.ca>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The FTP SIZE command allows us to get the size of a particular file,
as a consequence, we can now show proper transfer progression while
fetching a file using the FTP protocol.
Signed-off-by: Marin Hannache <git@mareo.fr>
Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[http] Treat any unexpected connection close as an error
iPXE currently checks that the server has not closed the connection
mid-stream (i.e. in the middle of a chunked transfer, or before the
specified Content-Length has been received), but does not check that
the server got as far as starting to send data. Consequently, if the
server closes the connection before any data is transferred (e.g. if
the server gives up waiting while iPXE performs the validation steps
for TLS), then iPXE will treat this as a successful transfer of a
zero-length file.
Fix by checking the RX connection state, and forcing an error if the
server has closed the connection at an unexpected point.
Originally-fixed-by: Marin Hannache <mareo@mareo.fr>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[http] Provide credentials only when requested by server
Provide HTTP Basic authentication credentials only in response to a
401 Unauthorized response from the server.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[http] Defer processing response code until after receiving all headers
Some headers can modify the meaning of the response code. For
example, a WWW-Authenticate header can change the interpretation of a
401 Unauthorized response from "Access denied" to "Please
authenticate".
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Michael Brown