[librm] Ensure that inline code symbols are unique
Commit 6149e0a ("[librm] Provide symbols for inline code placed into
other sections") may cause build failures due to duplicate label names
if the compiler chooses to duplicate inline assembly code.
Fix by using the "%=" special format string to include a
guaranteed-unique number within the label name.
The "%=" will be expanded only if constraints exist for the inline
assembly. This fix therefore requires that all REAL_CODE() fragments
use a (possibly empty) constraint list.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[librm] Provide symbols for inline code placed into other sections
Provide symbols constructed from the object name and line number for
code fragments placed into alternative sections, such as inline
REAL_CODE() assembly placed into .text16. This simplifies the
debugging task of finding the source code corresponding to a given
instruction pointer.
Note that we cannot use __FUNCTION__ since it is not a preprocessor
macro and so cannot be concatenated with string literals.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[librm] Add facility to provide register and stack dump for CPU exceptions
When DEBUG=librm_mgmt is enabled, intercept CPU exceptions and provide
a register and stack dump, then drop to an emergency shell. Exiting
from the shell will almost certainly not work, but this provides an
opportunity to view the register and stack dump and carry out some
basic debugging.
Note that we can intercept only the first 8 CPU exceptions, since a
PXE ROM is not permitted to rebase the PIC.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[librm] Add support for running in 64-bit long mode
Add support for running the BIOS version of iPXE in 64-bit long mode.
A 64-bit BIOS version of iPXE can be built using e.g.
make bin-x86_64-pcbios/ipxe.usb
make bin-x86_64-pcbios/8086100e.mrom
The 64-bit BIOS version should appear to function identically to the
normal 32-bit BIOS version. The physical memory layout is unaltered:
iPXE is still relocated to the top of the available 32-bit address
space. The code is linked to a virtual address of 0xffffffffeb000000
(in the negative 2GB as required by -mcmodel=kernel), with 4kB pages
created to cover the whole of .textdata. 2MB pages are created to
cover the whole of the 32-bit address space.
The 32-bit portions of the code run with VIRTUAL_CS and VIRTUAL_DS
configured such that truncating a 64-bit virtual address gives a
32-bit virtual address pointing to the same physical location.
The stack pointer remains as a physical address when running in long
mode (although the .stack section is accessible via the negative 2GB
virtual address); this is done in order to simplify the handling of
interrupts occurring while executing a portion of 32-bit code with
flat physical addressing via PHYS_CODE().
Interrupts may be enabled in either 64-bit long mode, 32-bit protected
mode with virtual addresses, 32-bit protected mode with physical
addresses, or 16-bit real mode. Interrupts occurring in any mode
other than real mode will be reflected down to real mode and handled
by whichever ISR is hooked into the BIOS interrupt vector table.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
In a 64-bit build, the entirety of the 32-bit address space is
identity-mapped and so any valid physical address may immediately be
used as a virtual address. Conversely, a virtual address that is
already within the 32-bit address space may immediately be used as a
physical address.
A valid virtual address that lies outside the 32-bit address space
must be an address within .textdata, and so can be converted to a
physical address by adding virt_offset.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[librm] Mark virt_offset, text16, data16, rm_cs, and rm_ds as constant
The physical locations of .textdata, .text16 and .data16 are constant
from the point of view of C code. Mark the relevant variables as
constant to allow gcc to optimise out redundant reads.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[librm] Add phys_call() wrapper for calling code with physical addressing
Add a phys_call() wrapper function (analogous to the existing
real_call() wrapper function) for calling code with flat physical
addressing, and use this wrapper within the PHYS_CODE() macro.
Move the relevant functionality inside librm.S, where it more
naturally belongs.
The COMBOOT code currently uses explicit calls to _virt_to_phys and
_phys_to_virt. These will need to be rewritten if our COMBOOT support
is ever generalised to be able to run in a 64-bit build.
Specifically:
- com32_exec_loop() should be restructured to use PHYS_CODE()
- com32_wrapper.S should be restructured to use an equivalent of
prot_call(), passing parameters via a struct i386_all_regs
- there appears to be no need for com32_wrapper.S to switch between
external and internal stacks; this could be omitted to simplify
the design.
For now, librm.S continues to expose _virt_to_phys and _phys_to_virt
for use by com32.c and com32_wrapper.S. Similarly, librm.S continues
to expose _intr_to_virt for use by gdbidt.S.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The bulk of the iPXE binary (the .textdata section) is physically
relocated at runtime to the top of the 32-bit address space in order
to allow space for an OS to be loaded. The relocation is achieved
with the assistance of segmentation: we adjust the code and data
segment bases so that the link-time addresses remain valid.
Segmentation is not available (for normal code and data segments) in
long mode. We choose to compile the C code with -mcmodel=kernel and
use a link-time address of 0xffffffffeb000000. This choice allows us
to identity-map the entirety of the 32-bit address space, and to alias
our chosen link-time address to the physical location of our .textdata
section. (This requires the .textdata section to always be aligned to
a page boundary.)
We simultaneously choose to set the 32-bit virtual address segment
bases such that the link-time addresses may simply be truncated to 32
bits in order to generate a valid 32-bit virtual address. This allows
symbols in .textdata to be trivially accessed by both 32-bit and
64-bit code.
There is no (sensible) way in 32-bit assembly code to generate the
required R_X86_64_32S relocation records for these truncated symbols.
However, subtracting the fixed constant 0xffffffff00000000 has the
same effect as truncation, and can be represented in a standard
R_X86_64_32 relocation record. We define the VIRTUAL() macro to
abstract away this truncation operation, and apply it to all
references by 32-bit (or 16-bit) assembly code to any symbols within
the .textdata section.
We define "virt_offset" for a 64-bit build as "the value to be added
to an address within .textdata in order to obtain its physical
address". With this definition, the low 32 bits of "virt_offset" can
be treated by 32-bit code as functionally equivalent to "virt_offset"
in a 32-bit build.
We define "text16" and "data16" for a 64-bit build as the physical
addresses of the .text16 and .data16 sections. Since a physical
address within the 32-bit address space may be used directly as a
64-bit virtual address (thanks to the identity map), this definition
provides the most natural access to variables in .text16 and .data16.
Note that this requires a minor adjustment in prot_to_real(), which
accesses .text16 using 32-bit virtual addresses.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[librm] Transition to protected mode within init_librm()
Long-mode operation will require page tables, which are too large to
sensibly fit in our .data16 segment in base memory.
Add a portion of init_librm() running in 32-bit protected mode to
provide access to high memory. Use this portion of init_librm() to
initialise the .textdata variables "virt_offset", "text16", and
"data16", eliminating the redundant (re)initialisation currently
performed on every mode transition as part of real_to_prot().
Signed-off-by: Michael Brown <mcb30@ipxe.org>
Move most arch/i386 files to arch/x86, and adjust the contents of the
Makefiles and the include/bits/*.h headers to reflect the new
locations.
This patch makes no substantive code changes, as can be seen using a
rename-aware diff (e.g. "git show -M5").
This patch does not make the pcbios platform functional for x86_64; it
merely allows it to compile without errors.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[legal] Relicense files under GPL2_OR_LATER_OR_UBDL
These files cannot be automatically relicensed by util/relicense.pl
since they either contain unusual but trivial contributions (such as
the addition of __nonnull function attributes), or contain lines
dating back to the initial git revision (and so require manual
knowledge of the code's origin).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[librm] Speed up protected-to-real mode transition under KVM
On an Intel CPU supporting VMX, KVM will emulate instructions while
the CPU state remains "invalid". In real mode, the CPU state is
defined to be "invalid" if any segment register has a base which is
not equal to (sreg<<4) or a limit which is not equal to 64kB.
We don't actually use the base stored in the REAL_DS descriptor for
any significant purpose. Change the base stored in this descriptor to
be equal to (REAL_DS<<4). A segment register loaded with REAL_DS is
then automatically valid in both real and protected modes. This
allows KVM to stop emulating instructions much sooner.
The only use of REAL_DS for memory accesses currently occurs in the
indirect ljmp within prot_to_real. Change this to a direct ljmp,
storing rm_cs in .text16 as part of the ljmp instruction. This
removes the only memory access via REAL_DS (thereby allowing for the
above descriptor base address hack), and also simplifies the ljmp
instruction (which will still have to be emulated).
Load the real-mode interrupt descriptor table register before
switching to real mode, since this avoids triggering an EXCEPTION_NMI
and corresponding VM exit.
This reduces the time taken by prot_to_real under KVM by around 65%.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
When running in a virtual machine, switching to real mode may be
expensive. Allow interrupts to be enabled while in protected mode and
reflected down to the real-mode interrupt handlers.
Signed-off-by: Michael Brown <mcb30@ipxe.org>
[librm] Use libflat to enable A20 line on each real-to-protected transition
Use the shared code in libflat to perform the A20 transitions
automatically on each transition from real to protected mode. This
allows us to remove all explicit calls to gateA20_set().
The old warnings about avoiding automatically enabling A20 are
essentially redundant; they date back to the time when we would always
start hammering the keyboard controller without first checking to see
if gate A20 was already enabled (which it almost always is).
Signed-off-by: Michael Brown <mcb30@ipxe.org>
The userptr_t is now the fundamental type that gets used for conversions.
For example, virt_to_phys() is implemented in terms of virt_to_user() and
user_to_phys().
[i386] Change semantics of __from_data16 and __from_text16
__from_data16 and __from_text16 now take a pointer to a
.data16/.text16 variable, and return the real-mode offset within the
appropriate segment. This matches the use case for every occurrence
of these macros, and prevents potential future bugs such as that fixed
in commit d51d80f. (The bug arose essentially because "&pointer" is
still syntactically valid.)
Add __bss16() macro, and allow use of .bss16 section by removing
link-time check for section overlaps. (In order to avoid wasting
space in the executable image, .bss16 will overlap with the following
section, which is .text).
Removed REAL_FRAGMENT and REAL_CALL, and left just REAL_EXEC (which is
the only one we actually use). This allows REAL_EXEC fragments to
contain proper references to constraints (e.g. "%w0"), rather than having
to force the use of specific registers.
Note that the "num_constraints" parameter is now completely obsolete, and
that we can probably reduce the syntax to something like
__asm__ __volatile__ ( REAL_CODE ( "asm statements" )
: output constraints
: input constraints
: clobber );
which would look much more natural, and avoid the need to always specify
a clobber list.
Add userptr_t to libkir.h, to allow it to at least compile.
Add the concept of a "user pointer" (similar to the void __user * in
the kernel), which encapsulates the information needed to refer to an
external buffer. Under normal operation, this can just be a void *
equivalent, but under -DKEEP_IT_REAL it would be a segoff_t equivalent.
Use this concept to avoid the need for bounce buffers in int13.c,
which reduces memory usage and opens up the possibility of using
multi-sector reads.
Extend the block-device API and the SCSI block device implementation
to support multi-sector reads.
Update iscsi.c to use user buffers.
Move the obsolete portions of realmode.h to old_realmode.h.
MS-DOS now boots an order of magnitude faster over iSCSI (~10 seconds
from power-up to C:> prompt in bochs).
Use .text16.data section with "aw" attributes, to avoid section type
conflicts when placing both code and data into .text16.
Add __from_{text16,data16}.
Michael Brown