[http] Rewrite HTTP core to support content encodings
Rewrite the HTTP core to allow for the addition of arbitrary content
encoding mechanisms, such as PeerDist and gzip.
The core now exposes http_open() which can be used to create requests
with an explicitly selected HTTP method, an optional requested content
range, and an optional request body. A simple wrapper provides the
preexisting behaviour of creating either a GET request or an
application/x-www-form-urlencoded POST request (if the URI includes
parameters).
The HTTP SAN interface is now implemented using the generic block
device translator. Individual blocks are requested using http_open()
to create a range request.
Server connections are now managed via a connection pool; this allows
for multiple requests to the same server (e.g. for SAN blocks) to be
completely unaware of each other. Repeated HTTPS connections to the
same server can reuse a pooled connection, avoiding the per-connection
overhead of establishing a TLS session (which can take several seconds
if using a client certificate).
Support for HTTP SAN booting and for the Basic and Digest
authentication schemes is now optional and can be controlled via the
SANBOOT_PROTO_HTTP, HTTP_AUTH_BASIC, and HTTP_AUTH_DIGEST build
configuration options in config/general.h.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Separate out the core HTTP functionality (which is shared by both HTTP
and HTTPS) from the provision of the "http://" URI opener. This
allows for builds that support only "https://" URIs.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
According to section 14.23 of RFC2616, an HTTP Host header without
port implies the default port is used. Thus, when fetching from
anywhere but port 80 for HTTP or 443 for HTTPS, the port ought to be
explicitly given in that header. Otherwise, some servers might fail
to associate the request with the correct virtual host or generate
incorrect self-referencing URLs.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Provide support for HTTP range requests, and expose this functionality
via the iPXE block device API. This allows SAN booting from a root
path such as:
sanboot http://boot.ipxe.org/freedos/fdfullcd.iso
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[process] Pass containing object pointer to process step() methods
Give the step() method a pointer to the containing object, rather than
a pointer to the process. This is consistent with the operation of
interface methods, and allows a single function to serve as both an
interface method and a process step() method.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Booting from an HTTP SAN will require HTTP range requests, which are
defined only in HTTP/1.1 and above. HTTP/1.1 mandates support for
"Transfer-Encoding: chunked", so we must support it.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[xfer] Generalise metadata "whence" field to "flags" field
iPXE has never supported SEEK_END; the usage of "whence" offers only
the options of SEEK_SET and SEEK_CUR and so is effectively a boolean
flag. Further flags will be required to support additional metadata
required by the Fibre Channel network model, so repurpose the "whence"
field as a generic "flags" field.
xfer_seek() has always been used with SEEK_SET, so remove the "whence"
field altogether from its argument list.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[interface] Convert all data-xfer interfaces to generic interfaces
Remove data-xfer as an interface type, and replace data-xfer
interfaces with generic interfaces supporting the data-xfer methods.
Filter interfaces (as used by the TLS layer) are handled using the
generic pass-through interface capability. A side-effect of this is
that deliver_raw() no longer exists as a data-xfer method. (In
practice this doesn't lose any efficiency, since there are no
instances within the current codebase where xfer_deliver_raw() is used
to pass data to an interface supporting the deliver_raw() method.)
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Standardise on using ref_init() to initialise an embedded reference
count, to match the coding style used by other embedded objects.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Access to the gpxe.org and etherboot.org domains and associated
resources has been revoked by the registrant of the domain. Work
around this problem by renaming project from gPXE to iPXE, and
updating URLs to match.
Also update README, LOG and COPYRIGHTS to remove obsolete information.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Commit 3d9dd93 introduced a regression in HTTP: if a URI without a
path is specified (e.g. http://netboot.me), we send the empty string
as our GET request. Reintroduce an extra slash when uri->path is NULL,
to turn this into the expected GET /.
Reported-by: Kyle Kienapfel <doctor.whom@gmail.com>
Signed-off-by: Joshua Oreman <oremanj@rwcr.net>
Signed-off-by: Marty Connor <mdc@etherboot.org>
Currently, handling of URI escapes is ad-hoc; escaped strings are
stored as-is in the URI structure, and it is up to the individual
protocol to unescape as necessary. This is error-prone and expensive
in terms of code size. Modify this behavior by unescaping in
parse_uri() and escaping in unparse_uri() those fields that typically
handle URI escapes (hostname, user, password, path, query, fragment),
and allowing unparse_uri() to accept a subset of fields to print so
it can be easily used to generate e.g. the escaped HTTP path?query
request.
Signed-off-by: Joshua Oreman <oremanj@rwcr.net>
Signed-off-by: Marty Connor <mdc@etherboot.org>
[xfer] Make consistent assumptions that xfer metadata can never be NULL
The documentation in xfer.h and xfer.c does not say that the metadata
parameter is optional in calls such as xfer_deliver_iob_meta() and the
deliver_iob() method. However, some code in net/ is prepared to
accept a NULL pointer, and xfer_deliver_as_iob() passes a NULL pointer
directly to the deliver_iob() method.
Fix this mess of conflicting assumptions by making everything assume
that the metadata parameter is mandatory, and fixing
xfer_deliver_as_iob() to pass in a dummy metadata structure (as is
already done in xfer_deliver_iob()).
[iobuf] Add iob_disown() and use it where it simplifies code
There are many functions that take ownership of the I/O buffer they
are passed as a parameter. The caller should not retain a pointer to
the I/O buffer. Use iob_disown() to automatically nullify the
caller's pointer, e.g.:
xfer_deliver_iob ( xfer, iob_disown ( iobuf ) );
This will ensure that iobuf is set to NULL for any code after the call
to xfer_deliver_iob().
iob_disown() is currently used only in places where it simplifies the
code, by avoiding an extra line explicitly setting the I/O buffer
pointer to NULL. It should ideally be used with each call to any
function that takes ownership of an I/O buffer. (The SSA
optimisations will ensure that use of iob_disown() gets optimised away
in cases where the caller makes no further use of the I/O buffer
pointer anyway.)
If gcc ever introduces an __attribute__((free)), indicating that use
of a function argument after a function call should generate a
warning, then we should use this to identify all applicable function
call sites, and add iob_disown() as necessary.
[http] gPXE is a HTTP/1.0 client, not a HTTP/1.1 client
gPXE is not compliant with the HTTP/1.1 specification (RFC 2616),
since it lacks support for "Transfer-Encoding: chunked". gPXE is,
however, compliant with the HTTP/1.0 specification (RFC 1945), which
does not require "Transfer-Encoding: chunked" to be supported.
The only HTTP/1.1 feature that gPXE uses is the "Host:" header, but
servers universally accept that one from HTTP/1.0 clients as an
optional extension (it is obligatory for HTTP/1.1). gPXE does not,
for example, appear to support connection caching. Advertising as a
HTTP/1.0 client will typically make the server close the connection
immediately upon sending the last data, which is actually beneficial
if we aren't going to keep the connection alive anyway.
Michael Brown