aboutsummaryrefslogtreecommitdiff
path: root/x86_64/locore.S
Commit message (Collapse)AuthorAgeFilesLines
* x86_64: fix double fault handlerLuca Dariz2024-09-081-1/+1
| | | | | | * x86_64/locore.S: adjust to the changes in the thread state structure (segment registers), and add the missing opcode. Message-ID: <20240904201806.510082-3-luca@orpolo.org>
* x86_64: split SET_KERNEL_SEGMENTS() for NCPU > 1Luca Dariz2024-03-091-3/+7
| | | | | | This allows 32on64 to work again. Also, it's a clearer indication of a missing part. Message-ID: <20240309140244.347835-1-luca@orpolo.org>
* x86_64 locore: Check segmentation by handSamuel Thibault2024-03-041-0/+20
| | | | | | | x86_64 ignores the segmentation limit, so we have to check it by hand when accessing userland pointers. Reported-by: Sergey Bugaev <bugaevc@gmail.com>
* locore: Remove unnecessary call to CPU_NUMBERDamien Zammit2024-02-181-2/+1
| | | | The cpu number is already in edx register, so use that.
* Fix 32-on-64: Fix missing es pushSamuel Thibault2023-12-031-1/+2
| | | | push %es actually cannot be compiled
* locore: Homogeneize 32bit and 64bitSamuel Thibault2023-10-281-0/+2
|
* 64bit: Fix locore buildSamuel Thibault2023-10-281-5/+5
| | | | | | | | | To allow references to int_stack_base to be quite unconstrained, we need to use 64bit register indexing. CPU_NUMBER_NO_GS was missing a 64bit variant. CPU_NUMBER_NO_STACK assumes being passed a 32bit register.
* x86_64/locore.S: Fix int stack checks when NCPUS > 1Damien Zammit2023-10-281-5/+8
| | | | Message-ID: <20231028001347.448826-1-damien@zamaudio.com>
* Factorize more push/pop codeSamuel Thibault2023-10-271-35/+34
|
* locore: Also factorize segment management on i386Samuel Thibault2023-10-271-22/+22
| | | | | | and harmonize i386/x86_64. This btw fixes not using dx in 32-on-64's alltraps.
* 32-on-64: Fix missing es pushSamuel Thibault2023-10-271-1/+1
|
* percpu: active_stack with gsDamien Zammit2023-09-251-1/+1
| | | | Message-Id: <20230925002417.467022-1-damien@zamaudio.com>
* x86_64: Change active_threads to use gs like i386Damien Zammit2023-09-251-4/+3
| | | | Message-Id: <20230925002353.466997-1-damien@zamaudio.com>
* start/switch_timer: Use CPU_NUMBERSamuel Thibault2023-09-241-2/+2
| | | | They are called from context that has gs initialized.
* percpu area using gs segmentDamien Zammit2023-09-241-3/+4
| | | | | | | | | | | | | | | This speeds up smp again, by storing the struct processor in a percpu area and avoiding an expensive cpu_number every call of current_processor(), as well as getting the cpu_number by an offset into the percpu area. Untested on 64 bit and work remains to use other percpu arrays. TESTED: (NCPUS=8) -smp 1 boots to login shell ~2x slower than uniprocessor TESTED: (NCPUS=8) -smp 2 boots to INIT but hangs there TESTED: (NCPUS=8) -smp 4 gets stuck seemingly within rumpdisk and hangs TESTED: (NCPUS=1) uniprocessor is a bit faster than normal Message-Id: <20230924103428.455966-3-damien@zamaudio.com>
* gdt: Cleanup gdt.h included in asmDamien Zammit2023-09-241-0/+1
| | | | Message-Id: <20230924052824.449219-2-damien@zamaudio.com>
* apic: Use cpuid to read the apic id for speedDamien Zammit2023-08-221-3/+3
| | | | Message-Id: <20230816014440.2322705-1-damien@zamaudio.com>
* IPI: Do not include support when NCPUS=1Samuel Thibault2023-08-131-0/+2
|
* IPI: Rework irq names and fix x86_64 buildSamuel Thibault2023-08-131-2/+2
|
* simple lock: check that the non-_irq variants are not called from IRQSamuel Thibault2023-08-121-3/+11
|
* x86_64: fix NCPUS > 1 build of CX() macroSamuel Thibault2023-08-121-39/+39
| | | | | With the kernel gone to -2GB, the base+index addressing needs to use a 64bit register index.
* x86_64: homogeneize with i386 about _call_singleSamuel Thibault2023-08-101-0/+4
|
* x86_64: fix recursive disabling of interruptsSamuel Thibault2023-08-101-2/+4
| | | | | In case interrupts were already disabled before TIME_TRAP_[US]ENTRY are called, we don't want to execute sti.
* x86_64: remove unneeded segment selectors handling on full 64 bitLuca Dariz2023-08-041-11/+5
| | | | | | | | | | | | | * i386/i386/db_interface.c: don't set unused segment selectors on full 64-bit * i386/i386/db_trace.c: likewise. * i386/i386/i386asm.sym: likewise. * i386/i386/pcb.c:: likewise. * i386/i386/thread.h: remove ES/DS/FS/GS from thread state on !USER32, as they are unused in this configuration. Only SS and CS are kept. * x86_64/locore.S: convert segment handling macros to no-op on full 64-bit Message-Id: <20230729174753.1145878-5-luca@orpolo.org>
* x86_64: refactor segment register handlingLuca Dariz2023-08-041-143/+85
| | | | | | | | | | | | | | | | | | | | | | | | | The actual values are not saved together with the rest of the thread state, both because it would be quite espensive (reading MSR, unless rdfsbase instructions are supported, but that's optional) and not really needed. The only way the user has to change its value is with a specific RPC, so we can intercept the change easily. Furthermore, Leaving the values there exposes them to being corrupted in case of a double interruption, e.g. an irq is handled just before iretq but after starting to restore the thread state. This solution was suggested by Sergey Bugaev. * i386/i386/db_trace.c: remove fsbase/gsbase from the registers available * i386/i386/debug_i386.c: remove fsbase/gsbase from the printed thread state * i386/i386/i386asm.sym: remove fsbase/gsbase as it's not needed in asm anymore * i386/i386/pcb.c: point fsbase/gsbase to the new location * i386/i386/thread.h: move fsbase/gsbase to the machine state * x86_64/locore.S: generalize segment-handling including es/ds/gs/fs and remove fsbase/gsbase handling. Also, factor out kernel segment selector setting to a macro. Message-Id: <20230729174753.1145878-4-luca@orpolo.org>
* x86_64: format pusha/popa macros for readabilityLuca Dariz2023-08-041-2/+35
| | | | Message-Id: <20230729174753.1145878-3-luca@orpolo.org>
* x86_64: disable V86 mode on full 64-bit configurationLuca Dariz2023-08-041-0/+8
| | | | | | | | * i386/i386/pcb.c: simplify exception stack location and adapt thread gettrs/setters * i386/i386/thread.h: don't include V86 fields on full 64-bit * x86_64/locore.S: don't include checks for V86 mode on full 64-bit Message-Id: <20230729174753.1145878-2-luca@orpolo.org>
* x86_64: fix stack handling on recursive interrupts for USER32Luca Dariz2023-08-041-5/+11
| | | | | | | | * x86_64/locore.S: ensure the thread state is filled completely even on recursive interrups. The value of the segment selectors is not very important in this case, but we still need to align the stack to the bottom of i386_interrupt_state. Message-Id: <20230729174753.1145878-1-luca@orpolo.org>
* x86_64: install emergency handler for double faultLuca Dariz2023-08-041-0/+15
| | | | | | | | | | | | * i386/i386/idt.c: add selector for the interrupt-specific stack * i386/i386/ktss.c: configure ist1 to use a dedicated stack * i386/i386/trap.c: add double fault handler, which just prints the state and panics. There is not much else to do in this case but it's useful for troubleshooting * x86_64/idt_inittab.S: allow to specify an interrupt stack for custom handlers * x86_64/locore.S: add double fault handler Message-Id: <20230729174514.1145656-1-luca@orpolo.org>
* x86_64: add a critical section on entry and exit from syscall/sysretLuca Dariz2023-06-171-1/+3
| | | | | | | | | | | When entering a syscall we're still using the user stack, so we can't reliably handle exceptions or interrupts, otherwise a user thread can easily crash the machine with an invalid stack. Instead, disable interrupts and (hopefullly) avoid traps in the fragments where we need to have the user stack in RSP. * i386/i386/ldt.c: mask interrupts and IOPL on syscall entry * x86_64/locore.S: keep interrupts disabled when we use the user stack
* fix fs/gs save/restore and USER32Luca Dariz2023-05-181-3/+1
| | | | | | * x86_64/locore.S: fix PUSH_FSGS -> _ISR and always make room for fsgsbase on a 64-bit kernel. Message-Id: <20230518210839.655403-1-luca@orpolo.org>
* x86_64: Attempt to fix ast_from_interruptSergey Bugaev2023-05-171-1/+2
| | | | Message-Id: <20230517181150.65810-1-bugaevc@gmail.com>
* x86_64: Check for AST when exiting a syscallSergey Bugaev2023-05-121-3/+25
| | | | | | | | | | ...like it's already done when exiting a trap. This is required, since handing a syscall can result in an AST; in particular this happens when the current thread is being terminated, which sets AST_TERMINATE and expects the thread to never return to userspace. Fixes a kernel crash upon calling exit () or pthread_exit () in glibc. Message-Id: <20230511192859.890693-1-bugaevc@gmail.com>
* add setting gs/fsbaseLuca Dariz2023-05-011-18/+71
| | | | | | | | | | | * i386/i386/i386asm.sym: add offsets for asm * i386/i386/pcb.c: switch FSBASE/GSBASE on context switch and implement accessors in thread setstatus/getstatus * i386/i386/thread.h: add new state to thread saved state * kern/thread.c: add i386_FSGS_BASE_STATE handler * x86_64/locore.S: fix fs/gs handling, skipping the base address and avoid resetting it by manually re-loading fs/gs Message-Id: <20230419194703.410575-5-luca@orpolo.org>
* x86_64: add 64-bit syscall entry pointLuca Dariz2023-05-011-5/+153
| | | | | | | | | | | | | | | | | | | | | | | | | While theoretically we could still use the same call gate as for 32-bit userspace, it doesn't seem very common, and gcc seems to not encode properly the instruction. Instead we use syscall/sysret as other kernels (e.g. XNU,Linux). This version still has some limitations, but should be enough to start working on the 64-bit user space. * i386/i386/i386asm.sym: add more constants to fill pcb->iss * i386/i386/ldt.c: configure 64-bit syscall entry point. We can just check for the SEP bit as MSR are always available on x86_64. * i386/i386/ldt.h: swap CS/DS segments order if !USER32 as required by sysret * i386/i386/locore.h: add syscall64 prototype * i386/i386/msr.h: add MSR definitions and C read/write helpers * i386/include/mach/i386/syscall_sw.h: remove old BSD_TRAP * x86_64/Makefrag.am: selectively install syscall_sw.h depending on USER32 * x86_64/include/syscall_sw.h: add entry point template from user space * x86_64/locore.S: implement syscall64 entry point and use it when a 64-bit user-space is configured Message-Id: <20230419194703.410575-4-luca@orpolo.org>
* fix address fault for 32-on-64-bit syscallLuca Dariz2023-04-201-2/+2
| | | | | | * x86_64/locore.S: the faulty address is found in %rbp and not in %rsi, so copy that in CR2 Message-Id: <20230419194703.410575-1-luca@orpolo.org>
* x86_64: fix user trap during syscall with an invalid user stackLuca Dariz2023-02-271-10/+10
| | | | | | | * i386/i386/locore.h: user vm_offset_t in the recovery_table * x86_64/locore.S: fix RECOVERY() location and keep user regs in %rbx, as it seems the convention. This only applies to 32-bit userspace. Message-Id: <20230227204501.2492152-5-luca@orpolo.org>
* extend data types to hold a 64-bit addressLuca Dariz2023-02-121-2/+2
| | | | | | | * i386/i386/trap.c: change from int to a proper type to hold a register value * x86_64/locore.S: use 64-bit register to avoid address truncation Message-Id: <20230212172818.1511405-8-luca@orpolo.org>
* fix x86_64 asm for higher kernel addressesLuca Dariz2023-02-121-4/+2
| | | | | | | * x86_64/interrupt.S: use 64-bit registers as variables could be stored at high addresses * x86_64/locore.S: Likewise Message-Id: <20230212172818.1511405-3-luca@orpolo.org>
* enable syscalls on x86_64Luca Dariz2023-02-121-3/+0
| | | | | Signed-off-by: Luca Dariz <luca@orpolo.org> Message-Id: <20230212170313.1501404-7-luca@orpolo.org>
* Fix other confusions between KERNEL_STACK_SIZE and INTSTACK_SIZESamuel Thibault2023-01-311-2/+2
|
* x86_64: expand and shrink messages in copy{in, out}msg routinesLuca Dariz2023-01-181-81/+0
| | | | | | | | | | | | | | | | | | | * i386/i386/copy_user.h: new file to handle 32/64 bit differences - add msg_usize() to recontruct the user-space message size - add copyin/copyout helpers for addresses and ports * include/mach/message.h: add msg alignment macros * ipc/ipc_kmsg.c: - copyin/out ports names instead of using pointer magic * ipc/ipc_mqueue.c: use msg_usize() to check if we can actually receive the message * ipc/mach_msg.c: Likewise for continuations in receive path * x86_64/Makefrag.am: add x86_64/copy_user.c * x86_64/copy_user.c: new file to handle message expansion and shrinking during copyinmsg/copyoutmsg for 64 bit kernels. - port names -> port pointers on all 64-bit builds - 32-bit pointer -> 64 bit pointer when using 32-bit userspace * x86_64/locore.S: remove copyinmsg() and copyoutmsg() Message-Id: <20230116105857.240210-3-luca@orpolo.org>
* x86_64: fix buildSamuel Thibault2022-11-271-1/+1
|
* Update kdb_kintr according to new interrupt stack layoutSamuel Thibault2022-11-271-2/+6
| | | | | | | | e2fcf261076b ("interrupt: Fix saving irq/ipl when linux drivers are disabled") changed the stack layout for interrupt handlers, but missed updating kdb_kintr's code that tries to mangle it. This restores the control-alt-d shortcut.
* x86_64: Add missing int_entry_table entries for APICSamuel Thibault2022-11-151-0/+15
| | | | | 214866d58ac2 ("Add ioapic support disabled by default") missed adding them for x86_64.
* x86_64: fix buildSamuel Thibault2022-09-171-1/+1
|
* copyinmsg: Set msgh_size inside copyinmsg rather than the callerSamuel Thibault2022-08-281-2/+4
| | | | | In the 32/64 conversion case it is copyinmsg that will know the eventual size.
* x86: Pave the way for xsave/xrstor supportSamuel Thibault2020-11-281-1/+5
| | | | | | | | | | | | | * i386/i386/fpu.h (xgetbv, get_xcr0, xsetbv, set_xcr0): New inline functions. (xsave, xrstor): New macros. * i386/i386/locore.S (cpu_features): Extend to two words. (cpu_features_edx, cpu_features_ecx): New labels. (discover_x86_cpu_type): Also save ecx cpuid(1) report. * x86_64/locore.S: Likewise. * i386/i386/locore.h (cpu_features): Extend to two words. (CPU_FEATURE_XSAVE): New macro. * i386/i386/proc_reg.h (CR4_OSXSAVE): New macro.
* mach_trap_table: Fix 64bit versionSamuel Thibault2020-04-061-1/+1
| | | | | | | The addition of the mach_trap_name field made the 64bit unused field spurious. * kern/syscall_sw.h (mach_trap_t): Remove `unused' field.
* x86_64: drop unused functionsSamuel Thibault2020-03-291-122/+0
| | | | | * x86_64/locore.S (dr6, dr0, dr1, dr2, dr3): Remove functions. (dr_msk, dr_addr): Remove variables.
generated by cgit v1.2.3 (git 2.39.1) at 2025-04-08 21:02:28 +0000