[build] Rewrite parserom.pl to support multiple source files
Running util/parserom.pl on all source files (637) one by one takes
approximately 35 seconds because of the startup cost of each invocation.
With the utility rewritten to support multiple source files it now takes
approximately 1 second to scan all source files for ROM declarations.
The --exclude-driver and --exclude-driver-class options have been added,
making it possible to skip certain source files from being scanned at all.
In addition --debug option has been added to more easily trace progress.
Finally --help option was added to show usage information.
Signed-off-by: Robin Smidsrød <robin@smidsrod.no>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
At some point in the past few years, binutils became more aggressive
at removing unused symbols. To function as a symbol requirement, a
relocation record must now be in a section marked with @progbits and
must not be in a section which gets discarded during the link (either
via --gc-sections or via /DISCARD/).
Update REQUIRE_SYMBOL() to generate relocation records meeting these
criteria. To minimise the impact upon the final binary size, we use
existing symbols (specified via the REQUIRING_SYMBOL() macro) as the
relocation targets where possible. We use R_386_NONE or R_X86_64_NONE
relocation types to prevent any actual unwanted relocation taking
place. Where no suitable symbol exists for REQUIRING_SYMBOL() (such
as in config.c), the macro PROVIDE_REQUIRING_SYMBOL() can be used to
generate a one-byte-long symbol to act as the relocation target.
If there are versions of binutils for which this approach fails, then
the fallback will probably involve killing off REQUEST_SYMBOL(),
redefining REQUIRE_SYMBOL() to use the current definition of
REQUEST_SYMBOL(), and postprocessing the linked ELF file with
something along the lines of "nm -u | wc -l" to check that there are
no undefined symbols remaining.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The UBDL relicensing tool (util/relicense.pl) is designed to identify
files which may be relicensed under a dual GPL+UBDL licence. It uses
git-blame to identify the author of each line (using the -M and -C
options to track lines moved or copied between files), and relicenses
files for which all authors have given permission.
The relicensing tool will ignore certain types of lines identified by
git-blame:
- empty lines
- comments
- standalone opening or closing braces
- "#include ..."
- "return 0;"
- "return rc;"
- "PCI_ROM(...)"
- "FILE_LICENCE(...)"
These lines either contain no meaningful content (e.g. empty lines),
contain only non-copyrightable facts (e.g. PCI ROM IDs) or are
sufficiently common within the codebase that git-blame is likely to
misattribute their origin (e.g. "return 0").
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[legal] Add support for the Unmodified Binary Distribution Licence
Add the text for the Unmodified Binary Distribution Licence. This
Licence allows for the distribution of unmodified binaries built from
publicly available source code, without imposing the obligations of
the GNU General Public License upon anyone who chooses to distribute
only the unmodified binaries built from that source code. See the
licence text for the precise terms and conditions.
Add the licence GPL2_OR_LATER_OR_UBDL to the set of licences which can
be declared using FILE_LICENCE(), and add the corresponding support to
licence.pl.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
LZMA provides significantly better compression (by ~15%) than the
current NRV2B algorithm.
We use a raw LZMA stream (aka LZMA1) to avoid the need for code to
parse the LZMA2 block headers. We use parameters {lc=2,lp=0,pb=0} to
reduce the stack space required by the decompressor to acceptable
levels (around 8kB). Using lc=3 or pb=2 would give marginally better
compression, but at the cost of substantially increasing the required
stack space.
The build process now requires the liblzma headers to be present on
the build system, since we do not include a copy of an LZMA compressor
within the iPXE source tree. The decompressor is written from scratch
(based on XZ Embedded) and is entirely self-contained within the
iPXE source.
The branch-call-jump (BCJ) filter used to improve the compressibility
is specific to iPXE. We choose not to use liblzma's built-in BCJ
filter since the algorithm is complex and undocumented. Our BCJ
filter achieves approximately the same results (on typical iPXE
binaries) with a substantially simpler algorithm.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
UEFI binaries may be relocated to any location within the 64-bit
address space. We compile as position-independent code with hidden
visibility, which should force all relocation records to be either
PC-relative (in which case no PE relocations are required) or full
64-bit relocations. There should be no R_X86_64_32 relocation
records, since that would imply an invalid assumption that code could
not be relocated above 4GB.
Remove support for R_X86_64_32 relocation records from util/elf2efi.c,
so that any such records result in a build failure rather than a
potential runtime failure.
Reported-by: Jan Kundrát <jkt@kde.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Add utility for constructing EFI fat binaries (dual 32/64-bit
binaries, usable only on Apple EFI systems).
This utility is not part of the standard build process. To use it:
make util/efifatbin bin-i386-efi/ipxe.efi bin-x86_64-efi/ipxe.efi
and then
./util/efifatbin bin-*-efi/ipxe.efi fat-ipxe.efi
Requested-by: Brandon Penglase <bpenglase-ipxe@spaceservices.net>
Tested-by: Brandon Penglase <bpenglase-ipxe@spaceservices.net>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[util] Use PCI length field to obtain length of individual images
Option::ROM currently uses the initialisation length field (single
byte at offset 0x02) to determine the length of a ROM image within a
multi-image ROM file. For PCI ROM images with a code type other than
0, the initialisation length field may not be present.
Fix by using the PCI header's image length field instead. Note that
this does not prevent us from correctly handling ISA ROMs, since ISA
ROMs do not support multiple images within a single ROM BAR anyway.
Inspired-by: Swift Geek <swiftgeek@gmail.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Rework geniso and genliso to provide a single merged utility for
generating ISO images.
Modified-by: Michael Brown <mcb30@ipxe.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Syslinux 6.x places its files into a bios subdirectory, and requires
that a ldlinux.c32 module be included within the ISO image. Add the
relevant search paths for isolinux.bin, and include the file
ldlinux.c32 within the ISO image if it exists.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[libc] Redefine low 8 bits of error code as "platform error code"
The low 8 bits of an iPXE error code are currently defined as the
closest equivalent PXE error code. Generalise this scheme to
platforms other than PC-BIOS by extending this definition to "closest
equivalent platform error code". This allows for the possibility of
returning meaningful errors via EFI APIs.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Newer versions of bfd.h require definitions for the PACKAGE and
PACKAGE_VERSION macros used by autotools. Work around this by
manually defining these macros before including bfd.h.
Originally-fixed-by: Brandon Penglase <bpenglase-ipxe@spaceservices.net>
Tested-by: Brandon Penglase <bpenglase-ipxe@spaceservices.net>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[zbin] Fix size used for memset in alloc_output_file
The output->buf field is a pointer, not an array, so sizeof() is not
applicable. We must use the allocated string length instead.
Identified by gcc:
util/zbin.c: In function ‘alloc_output_file’:
util/zbin.c:146:37: warning: argument to ‘sizeof’ in ‘memset’ call
is the same expression as the destination; did you mean to
dereference it? [-Wsizeof-pointer-memaccess]
memset ( output->buf, 0xff, sizeof ( output->buf ) );
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[efi] Ensure EFI binaries comply with Authenticode requirements
Authenticode requires that the size of the raw file must equal the
size of the OptionalHeader.SizeOfHeaders plus the sum of all sections'
SizeOfRawData.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[util] Fix up checksum in UNDI ROM header, if present
The UNDI ROM header does contain a checksum byte. Apparently no-one
cares about this, since iPXE has left it as zero for years without
anyone noticing.
Since Option::ROM now understands the UNDI ROM header, we may as well
fix up the checksum byte for the sake of completeness.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[util] Allow for CALL NEAR in the option ROM initialisation entry point
Option::ROM currently understands only JMP NEAR and JMP SHORT
instructions in the initialisation entry point. At least one Broadcom
option ROM has been observed to use a CALL NEAR instruction.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[romprefix] Report a pessimistic runtime size estimate
PCI3.0 allows us to report a "runtime size" which can be smaller than
the actual ROM size. On systems that support PMM our runtime size
will be small (~2.5kB), which helps to conserve the limited option ROM
space. However, there is no guarantee that the PMM allocation will
succeed, and so we need to report the worst-case runtime size in the
PCI header.
Move the "shrunk ROM size" field from the PCI header to a new "iPXE
ROM header", allowing it to be accessed by ROM-manipulation utilities
such as disrom.pl.
Reported-by: Anton D. Kachalov <mouse@yandex-team.ru>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Commit 196751c ("[build] Enable warnings when building utilities")
revealed a previously hidden compiler warning in util/nrv2b.c
regarding an out-of-bounds array subscript in the code
#if defined(SWD_BEST_OFF)
if (s->best_pos[2] == 0)
s->best_pos[2] = key + 1;
#endif
where best_pos[] is defined by
#define SWD_BEST_OFF 1
#if defined(SWD_BEST_OFF)
unsigned int best_off[ SWD_BEST_OFF ];
unsigned int best_pos[ SWD_BEST_OFF ];
#endif
With SWD_BEST_OFF set to 1, it can be proven that all code paths
referring to s->best_off[] and s->best_pos[] will never be executed,
with the exception of the two lines above. Since these two lines
alone can have no effect on execution, we can safely undefine
SWD_BEST_OFF.
Verified by comparing md5sums of bin/undionly.kpxe before and after
the change.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[romprefix] Add a dummy ROM header to cover the .mrom payload
The header of a .mrom image declares its length to be only a few
kilobytes; the remainder is accessed via a sideband mechanism. This
makes it difficult to append an additional ROM image, such as an EFI
ROM.
Add a second, dummy ROM header covering the payload portion of the
.mrom image, allowing consumers to locate any appended ROM images in
the usual way.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[util] Remove obsolete Makefile rule for util/prototester.c
util/prototester.c was removed in commit a6d1815 ("Obsolete for some
time now") back in 2006.
Signed-off-by: Marin Hannache <mareo@mareo.fr>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[util] Add utility to generate list of supported network cards
niclist.pl recursively scans specified source folders and builds a
list of supported NICs by looking for ISA_ROM and PCI_ROM lines and
outputs the list in text, CSV, JSON, HTML or DokuWiki format. Sorting
and column selection is possible.
The pci-utils pci.ids file is fetched from SourceForge once a day to
also output the "official" vendor/device names associated with the PCI
device.
Signed-off-by: Robin Smidsrød <robin@smidsrod.no>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[console] Ignore unexpected keysyms when generating keyboard maps
I am unable to find any definitive documentation on how Linux keyboard
symbols work. In the absence of any documentation, I'm going to
assume that unexpected keysyms are harmless and should be ignored.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[build] Include UNDI PCI driver within all-drivers build
Commit 9b99d2a ("[build] Avoid generating ROMs with "match-any" vendor
or device IDs") introduced a regression which caused the UNDI PCI
driver to be omitted from the list of all drivers, and thus to be
excluded from the all-drivers build.
Fix by ensuring that the per-driver section of the Makefile is
generated even when there are no ROMs to be built.
Reported-by: Sven Dreyer <sven@dreyer-net.de>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Provide a utility to quickly determine the ROM size and .mrom format
support for attached PCI devices. For example:
01:00.0 (1186:4300) supports a 128kB .rom or .mrom
Inspired-by: Wes Frazier <wes.frazier@members.fsf.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[build] Avoid generating ROMs with "match-any" vendor or device IDs
A PCI_ROM() entry containing a vendor or device ID of PCI_ANY_ID
(0xffff) indicates to pci_find_driver() that the entry's vendor or
device ID should be ignored when matching against the device's vendor
or device ID. It does not represent a PCI ROM that should be built.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[build] Include only one copy of each ROM in "make allroms"
Each PCI ROM currently ends up appearing twice in the $(ROMS) list:
once under its designated name (e.g. "rtl8139.rom"), once under its
PCI IDs (e.g. "bin/10ec8139.rom").
Include only the latter of these in the $(ROMS) list, so that doing
"make allroms" will generate only one copy of each ROM.
Reported-by: Bastian Blank <waldi@debian.org>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[console] Try to avoid problems caused by keycode 86
The "us" keyboard layout contains a mapping for keycode 86 (which
seems not to correspond to any physical key on many US keyboards) to
the ASCII character '<'. This mapping causes conflicts with the
mapping for keycode 51, which also maps (with shift) to '<'.
Change the keyboard mapping generator to choose the lowest keycode for
each ASCII character as indicating the relevant mapping to use, on the
basis that a lower keycode roughly indicates a "more normal" key. On
a German keyboard, which has keys for both keycode 51 and keycode 86
present, this causes '<' to be remapped to ';', which is a closer
match to typical user expectations.
Reported-by: Sven Dreyer <sven@dreyer-net.de>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Inspired by LILO's keytab-lilo.pl, genkeymap.pl uses "loadkeys -b" to
obtain a Linux keyboard map, and generates a file keymap_xx.c in
hci/keymap.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Enhance the information collected by the function recorder to include
the call site and entry/exit counts. This allows fnrec.pl to produce
a call tree such as:
step (from core/getkey.c:46 = 0x17e90) {
ref_increment (from core/process.c:93 = 0x73ec) { }
net_step (from core/process.c:96 = 0x73f1) {
net_poll (from net/netdevice.c:741 = 0xbce6) {
netdev_poll (from net/netdevice.c:700 = 0xbc58) { }
netdev_rx_dequeue (from net/netdevice.c:709 = 0xbc65) { }
}
}
ref_decrement (from core/process.c:96 = 0x73f9) { }
}
Note that inlined functions are reported, confusingly, as extra calls
to the *containing* function. Minimise this confusion by adding the
attribute "no_instrument_function" to all functions declared as
inline. (Static functions that have been inlined autonomously by gcc
will still be problematic, but these are far fewer in number.)
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Currently, if elf2efi.c is compiled using a 32-bit HOST_CC, then the
resulting elf2efi64 binary will generate 32-bit EFI binaries instead
of 64-bit EFI binaries.
The problem is that elf2efi.c uses the MDE_CPU_* definitions to decide
whether to output a 32-bit or 64-bit PE binary. However, MDE_CPU_*
gets defined in ProcessorBind.h, depending on the compiler's target
architecture. Overriding them on the command line doesn't work in the
expected way, and you can end up in cases where both MDE_CPU_IA32 and
MDE_CPU_X64 are defined.
Fix by using a separate definition, EFI_TARGET_IA32/EFI_TARGET_X64,
which is specified only on the command line.
Signed-off-by: Geoff Lywood <glywood@vmware.com>
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Currently, if you attempt to build 64-bit EFI binaries on a 32-bit
system without a suitable cross-compiling version of libbfd, the iPXE
build will die with a segmentation fault in elf2efi64.
Fix by properly handling the return value from bfd_check_format().
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[romprefix] Add .mrom format, allowing loading of large ROMs
Add an infrastructure allowing the prefix to provide an open_payload()
method for obtaining out-of-band access to the whole iPXE image. Add
a mechanism within this infrastructure that allows raw access to the
expansion ROM BAR by temporarily borrowing an address from a suitable
memory BAR on the same PCI card.
For cards that have a memory BAR that is at least as large as their
expansion ROM BAR, this allows large iPXE ROMs to be supported even on
systems where PMM fails, or where option ROM space pressure makes it
impossible to use PMM shrinking. The BIOS sees only a stub ROM of
approximately 3kB in size; the remainder (which can be well over 64kB)
is loaded only at the time iPXE is invoked.
As a nice side-effect, an iPXE .mrom image will continue to work even
if its PMM-allocated areas are overwritten between initialisation and
invocation.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[build] Replace obsolete makerom.pl with quick script using Option::ROM
The only remaining useful function of makerom.pl is to correct the ROM
and PnP checksums; the PCI IDs are set at link time, and padding is
performed using padimg.pl.
Option::ROM already provides a facility for correcting the checksums,
so we may as well just use this instead.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Add a section .text16.early which is always kept inline with the
prefix. This will allow for some code sharing between the .prefix and
.text16 sections.
Note that the simple solution of just prepending the .prefix section
to the .text16 section will not work, because a bug in Wyse Streaming
Manager server (WLDRM13.BIN) requires us to place a dummy PXENV+ entry
point at the start of .text16.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Michael Brown