LZMA provides significantly better compression (by ~15%) than the
current NRV2B algorithm.

We use a raw LZMA stream (aka LZMA1) to avoid the need for code to
parse the LZMA2 block headers.  We use parameters {lc=2,lp=0,pb=0} to
reduce the stack space required by the decompressor to acceptable
levels (around 8kB).  Using lc=3 or pb=2 would give marginally better
compression, but at the cost of substantially increasing the required
stack space.

The build process now requires the liblzma headers to be present on
the build system, since we do not include a copy of an LZMA compressor
within the iPXE source tree.  The decompressor is written from scratch
(based on XZ Embedded) and is entirely self-contained within the
iPXE source.

The branch-call-jump (BCJ) filter used to improve the compressibility
is specific to iPXE.  We choose not to use liblzma's built-in BCJ
filter since the algorithm is complex and undocumented.  Our BCJ
filter achieves approximately the same results (on typical iPXE
binaries) with a substantially simpler algorithm.

Signed-off-by: Michael Brown <mcb30@ipxe.org>

The output->buf field is a pointer, not an array, so sizeof() is not
applicable.  We must use the allocated string length instead.

Identified by gcc:

  util/zbin.c: In function ‘alloc_output_file’:
  util/zbin.c:146:37: warning: argument to ‘sizeof’ in ‘memset’ call
    is the same expression as the destination; did you mean to
    dereference it? [-Wsizeof-pointer-memaccess]
  memset ( output->buf, 0xff, sizeof ( output->buf ) );

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>

The header of a .mrom image declares its length to be only a few
kilobytes; the remainder is accessed via a sideband mechanism.  This
makes it difficult to append an additional ROM image, such as an EFI
ROM.

Add a second, dummy ROM header covering the payload portion of the
.mrom image, allowing consumers to locate any appended ROM images in
the usual way.

Signed-off-by: Michael Brown <mcb30@ipxe.org>

Signed-off-by: Michael Brown <mcb30@ipxe.org>

Add a section .text16.early which is always kept inline with the
prefix.  This will allow for some code sharing between the .prefix and
.text16 sections.

Note that the simple solution of just prepending the .prefix section
to the .text16 section will not work, because a bug in Wyse Streaming
Manager server (WLDRM13.BIN) requires us to place a dummy PXENV+ entry
point at the start of .text16.

Signed-off-by: Michael Brown <mcb30@ipxe.org>

Recent gcc versions generate more warnings when compiling util/zbin.c
on a 64-bit system:

util/zbin.c: In function `read_file':
util/zbin.c:85: warning: format `%d' expects type `int', but
                argument 3 has type `size_t'
util/zbin.c:91: warning: format `%d' expects type `int', but
                argument 3 has type `size_t'
util/zbin.c: In function `read_zinfo_file':
util/zbin.c:119: warning: format `%d' expects type `int', but
                 argument 4 has type `size_t'
util/zbin.c: In function `alloc_output_file':
util/zbin.c:134: warning: format `%d' expects type `int', but
                 argument 3 has type `size_t'
util/zbin.c: In function `process_zinfo_add':
util/zbin.c:244: warning: format `%d' expects type `int', but
                 argument 3 has type `size_t'
util/zbin.c:266: warning: format `%d' expects type `int', but
                 argument 7 has type `size_t'
util/zbin.c:286: warning: format `%#x' expects type `unsigned int',
                 but argument 7 has type `size_t'
util/zbin.c: In function `write_output_file':
util/zbin.c:348: warning: format `%d' expects type `int', but
                 argument 3 has type `size_t'

This patch eliminates these warnings.

Signed-off-by: Marty Connor <mdc@etherboot.org>

Recent gcc versions generate warnings when compiling util/zbin.c
( tested with gcc-4.3.3 ):

util/zbin.c: In function ‘process_zinfo_pack’:
util/zbin.c:200: warning: format ‘%#zx’ expects type ‘size_t’, but argument 6
has type ‘long unsigned int’
util/zbin.c: In function ‘process_zinfo_add’:
util/zbin.c:257: warning: format ‘%#lx’ expects type ‘long unsigned int’, but
argument 4 has type ‘int’
util/zbin.c:266: warning: format ‘%#lx’ expects type ‘long unsigned int’, but
argument 4 has type ‘int’
util/zbin.c:266: warning: format ‘%d’ expects type ‘int’, but argument 8 has
type ‘long unsigned int’
util/zbin.c:286: warning: format ‘%#lx’ expects type ‘long unsigned int’, but
argument 6 has type ‘int’
util/zbin.c:286: warning: format ‘%#lx’ expects type ‘long unsigned int’, but
argument 7 has type ‘size_t’

This patch eliminates these warnings.

Tested with gcc-4.3.3 on Ubuntu 9.04 and gcc-4.1.2 on Debian Etch.

Signed-off-by: Marty Connor <mdc@etherboot.org>

The option ROM header contains a one-byte field indicating the number
of 512-byte sectors in the ROM image.  Currently it is linked to
contain the number of uncompressed sectors, with an instruction to the
compressor to correct it.  This causes link failure when the
uncompressed size of the ROM image is over 128k.

Fix by replacing the SUBx compressor fixup with an ADDx fixup that
adds the total compressed output length, scaled as requested, to an
addend stored in the field where the final length value will be
placed.  This is similar to the behavior of ELF relocations, and
ensures that an overflow error will not be generated unless the
compressed size is still too large for the field.

This also allows us to do away with the _filesz_pgh and _filesz_sect
calculations exported by the linker script.

Output tested bitwise identical to the old SUBx mechanism on hd, dsk,
lkrn, and rom prefixes, on both 32-bit and 64-bit processors.

Modified-by: Michael Brown <mcb30@etherboot.org>
Signed-off-by: Michael Brown <mcb30@etherboot.org>

Commit b149a99 ([build] Round up SUBx deltas) introduced a
signed/unsigned issue that affects gPXE images built on 32-bit hosts.
The zbin fixup utility performed an unsigned division, which led to
.usb images with an incorrect number of sectors to load.

The issue snuck by on 64-bit hosts since uint32_t is promoted to long.
On 32-bit hosts it is promoted to unsigned long.

Modified-by: Michael Brown <mcb30@etherboot.org>
Signed-off-by: Michael Brown <mcb30@etherboot.org>

The zbin compressor fixup utility rounds down file sizes before
calculating their difference.  This produces incorrect values and may
cause truncated gPXE images to be loaded at boot.

The following example explains the problem:
ilen    = 48 bytes     (uncompressed input file)
olen    = 17 bytes     (compressed output file)
divisor = 16 bytes     (paragraph granularity)
fixmeup = 3 paragraphs (value to fix up)

olen / divisor - ilen / divisor
= 1 - 3
= -2 paragraphs  (old delta calculation)

( align ( olen, divisor ) - align ( ilen, divisor ) ) / divisor
= 2 - 3
= -1 paragraphs  (new delta calculation)

If we perform the SUBx operation with old delta:
fixmeup + -2 = 1 paragraph gets loaded by the prefix

With the new delta:
fixmeup + -1 = 2 paragraphs get loaded by the prefix

The old delta calculation removes the last paragraph; the prefix will
load a truncated copy of gPXE into memory.  We need to load 2
paragraphs since olen is 17 bytes.  Loading only 1 paragraph (16
bytes) would truncate the last byte.

Signed-off-by: Michael Brown <mcb30@etherboot.org>

Must check that argument to a fclose() is not NULL -- we can get to the
'err' label when file was not opened.  fclose(NULL) is known to produce
core dump on some platforms and we don't want zbin to fail so loudly.

Signed-off-by: Eygene Ryabinkin <rea-fbsd@codelabs.ru>