summaryrefslogtreecommitdiff
path: root/src/arch/arm64/armv8/mmu.c
AgeCommit message (Collapse)Author
2020-03-06src/arch/arm64: Convert to SPDX license headerPatrick Georgi
This also drops individual copyright notices, all mentioned authors in that part of the tree are already listed in AUTHORS. Change-Id: Ic5eddc961d015328e5a90994b7963e7af83cddd3 Signed-off-by: Patrick Georgi <pgeorgi@google.com> Reviewed-on: https://review.coreboot.org/c/coreboot/+/39279 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net> Reviewed-by: HAOUAS Elyes <ehaouas@noos.fr> Reviewed-by: Angel Pons <th3fanbus@gmail.com>
2019-12-19src/arch: Remove unused <stdlib.h>Elyes HAOUAS
Change-Id: I79f065703b5249ca9630b06de7142bc52675076e Signed-off-by: Elyes HAOUAS <ehaouas@noos.fr> Reviewed-on: https://review.coreboot.org/c/coreboot/+/32820 Reviewed-by: Kyösti Mälkki <kyosti.malkki@gmail.com> Tested-by: build bot (Jenkins) <no-reply@coreboot.org>
2018-08-13arm64: mmu: Spot check security state for TTB mappingJulius Werner
Since commit 372d0ff1d1 (arch/arm64: mmu: Spot check TTB memory attributes), we already check the memory attributes that the TTB region is mapped with to avoid configuration mistakes that cause weird issues (because the MMU walks the page tables with different memory attributes than they were written with). Unfortunately, we only checked cachability, but the security state attribute is just as important for this (because it is part of the cache tag, meaning that a cache entry created by accessing the non-secure mapping won't be used when trying to read the same address through a secure mapping... and since AArch64 page table walks are cache snooping and we rely on that behavior, this can lead to the MMU not seeing the new page table entries we just wrote). This patch adds the check for security state and cleans up that code a little. Change-Id: I70cda4f76f201b03d69a9ece063a3830b15ac04b Signed-off-by: Julius Werner <jwerner@chromium.org> Reviewed-on: https://review.coreboot.org/28017 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Hung-Te Lin <hungte@chromium.org> Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net>
2018-08-10arm64: Drop checks for current exception level, hardcode EL3 assumptionJulius Werner
When we first created the arm64 port, we weren't quite sure whether coreboot would always run in EL3 on all platforms. The AArch64 A.R.M. technically considers this exception level optional, but in practice all SoCs seem to support it. We have since accumulated a lot of code that already hardcodes an implicit or explicit assumption of executing in EL3 somewhere, so coreboot wouldn't work on a system that tries to enter it in EL1/2 right now anyway. However, some of our low level support libraries (in particular those for accessing architectural registers) still have provisions for running at different exception levels built-in, and often use switch statements over the current exception level to decide which register to access. This includes an unnecessarily large amount of code for what should be single-instruction operations and precludes further optimization via inlining. This patch removes any remaining code that dynamically depends on the current exception level and makes the assumption that coreboot executes at EL3 official. If this ever needs to change for a future platform, it would probably be cleaner to set the expected exception level in a Kconfig rather than always probing it at runtime. Change-Id: I1a9fb9b4227bd15a013080d1c7eabd48515fdb67 Signed-off-by: Julius Werner <jwerner@chromium.org> Reviewed-on: https://review.coreboot.org/27880 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Patrick Rudolph <patrick.rudolph@9elements.com> Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2018-06-26arm64: Reimplement mmu_disable() in assemblyJulius Werner
Disabling the MMU with proper cache behavior is a bit tricky on ARM64: you can flush the cache first and then disable the MMU (like we have been doing), but then you run the risk of having new cache lines allocated in the tiny window between the two, which may or may not become a problem when those get flushed at a later point (on some platforms certain memory regions "go away" at certain points in a way that makes the CPU very unhappy if it ever issues a write cycle to them again afterwards). The obvious alternative is to first disable the MMU and then flush the cache, ensuring that every memory access after the flush already has the non-cacheable attribute. But we can't just flip the order around in the C code that we have because then those accesses in the tiny window in-between will go straight to memory, so loads may yield the wrong result or stores may get overwritten again by the later cache flush. In the end, this all shouldn't really be a problem because we can do both operations purely from registers without doing any explicit memory accesses in-between. We just have to reimplement the function in assembly to make sure the compiler doesn't insert any stack accesses at the wrong points. Change-Id: Ic552960c91400dadae6f130b2521a696eeb4c0b1 Signed-off-by: Julius Werner <jwerner@chromium.org> Reviewed-on: https://review.coreboot.org/27238 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2018-04-30arm64: Add mmu context save/restore APIsT Michael Turney
New API required by sdm845 DDR init/training protocol TEST=build & run Change-Id: I8442442c0588dd6fb5e461b399e48a761f7bbf29 Signed-off-by: T Michael Turney <mturney@codeaurora.org> Reviewed-on: https://review.coreboot.org/25818 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2018-03-23arch/arm64/armv8/mmu: Add support for 48bit VAPatrick Rudolph
The VA space needs to be extended to support 48bit, as on Cavium SoCs the MMIO starts at 1 << 47. The following changes were done to coreboot and libpayload: * Use page table lvl 0 * Increase VA bits to 48 * Enable 256TB in MMU controller * Add additional asserts Tested on Cavium SoC and two ARM64 Chromebooks. Change-Id: I89e6a4809b6b725c3945bad7fce82b0dfee7c262 Signed-off-by: Patrick Rudolph <patrick.rudolph@9elements.com> Reviewed-on: https://review.coreboot.org/24970 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Julius Werner <jwerner@chromium.org>
2017-05-30arm64: Align cache maintenance code with libpayload and ARM32Julius Werner
coreboot and libpayload currently use completely different code to perform a full cache flush on ARM64, with even different function names. The libpayload code is closely inspired by the ARM32 version, so for the sake of overall consistency let's sync coreboot to that. Also align a few other cache management details to work the same way as the corresponding ARM32 parts (such as only flushing but not invalidating the data cache after loading a new stage, which may have a small performance benefit). Change-Id: I9e05b425eeeaa27a447b37f98c0928fed3f74340 Signed-off-by: Julius Werner <jwerner@chromium.org> Reviewed-on: https://review.coreboot.org/19785 Tested-by: build bot (Jenkins) <no-reply@coreboot.org> Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2016-02-10arch/arm64: mmu: Spot check TTB memory attributesJulius Werner
On ARM64, the memory type for accessing page table descriptors during address translation is governed by the Translation Control Register (TCR). When the MMU code accesses the same descriptors to change page mappings, it uses the standard memory type rules (defined by the page table descriptor for the page that contains that table, or 'device' if the MMU is off). Accessing the same memory with different memory types can lead to all kinds of fun and hard to debug effects. In particular, if the TCR says "cacheable" and the page tables say "uncacheable", page table walks will pull stale entries into the cache and later mmu_config_range() calls will write directly to memory, bypassing those cache lines. This means the translations will not get updated even after a TLB flush, and later cache flushes/evictions may write the stale entries back to memory. Since page table configuration is currently always done from SoC code, we can't generally ensure that the TTB is always mapped as cacheable. We can however save developers of future SoCs a lot of headaches and time by spot checking the attributes when the MMU gets enabled, as this patch does. BRANCH=None BUG=None TEST=Booted Oak. Manually tested get_pte() with a few addresses. Change-Id: I3afd29dece848c4b5f759ce2f00ca2b7433374da Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: f3947f4bb0abf4466006d5e3a962bbcb8919b12d Original-Change-Id: I1008883e5ed4cc37d30cae5777a60287d3d01af0 Original-Signed-off-by: Julius Werner <jwerner@chromium.org> Original-Reviewed-on: https://chromium-review.googlesource.com/323862 Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: https://review.coreboot.org/13595 Tested-by: build bot (Jenkins) Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-11-11arm64: mmu: Make page table manipulation work across stagesJulius Werner
In order to have a proper runtime-modifyable page table API (e.g. to remap DRAM after it was intialized), we need to remove any external bookkeeping kept in global variables (which do not persist across stages) from the MMU code. This patch implements this in a similar way as it has recently been done for ARM32 (marking free table slots with a special sentinel value in the first PTE that cannot occur as part of a normal page table). Since this requires the page table buffer to be known at compile-time, we have to remove the option of passing it to mmu_init() at runtime (which I already kinda deprecated before). The existing Tegra chipsets that still used it are switched to instead define it in memlayout in a minimally invasive change. This might not be the best way to design this overall (I think we should probably just throw the tables into SRAM like on all other platforms), but I don't have a Tegra system to test so I'd rather keep this change low impact and leave the major redesign for later. Also inlined some single-use one-liner functions in mmu.c that I felt confused things more than they cleared up, and fixed an (apparently harmless?) issue with forgetting to mask out the XN page attribute bit when casting a table descriptor to a pointer. BRANCH=None BUG=None TEST=Compiled Ryu and Smaug. Booted Oak. Change-Id: Iad71f97f5ec4b1fc981dbc8ff1dc88d96c8ee55a Signed-off-by: Julius Werner <jwerner@chromium.org> Reviewed-on: http://review.coreboot.org/12075 Tested-by: build bot (Jenkins) Reviewed-by: Aaron Durbin <adurbin@chromium.org>
2015-09-28arm64: mmu: Prevent CPU prefetch instructions from device memoryJimmy Huang
Set XN bit of block upper attribute to device memory in mmu. CPU may speculatively prefetch instructions from device memory, but the IO subsystem of some implementation may not support this operation. Set this attribute to device memory mmu entries can prevent CPU from prefetching device memory. BRANCH=none BUG=none TEST=build and booted to kernel on oak-rev3 with dcm enabled. Change-Id: I52ac7d7c84220624aaf6a48d64b9110d7afeb293 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: 7b01a4157cb046a5e75ea7625060a602e7a63c3c Original-Change-Id: Id535e990a23b6c89123b5a4e64d7ed21eebed607 Original-Signed-off-by: Jimmy Huang <jimmy.huang@mediatek.com> Original-Reviewed-on: https://chromium-review.googlesource.com/302301 Original-Commit-Ready: Yidi Lin <yidi.lin@mediatek.com> Original-Tested-by: Yidi Lin <yidi.lin@mediatek.com> Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org> Original-Reviewed-by: Julius Werner <jwerner@chromium.org> Reviewed-on: http://review.coreboot.org/11722 Tested-by: build bot (Jenkins) Reviewed-by: Patrick Georgi <pgeorgi@google.com>
2015-06-02arm64: correct cacheable/non-cacheable tag string in print_tagJimmy Huang
BRANCH=none BUG=none TEST=Booted on Oak and confirmed the output cacheable/non-cacheable string is correct. Change-Id: I062c1cc384b8cb9d07038399b1bc7ef47d992103 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: 45552f95b55cd9fc81817e4ff02c78e885377065 Original-Change-Id: Ie52066dbefd2f54d0746792b89f0b57767811adb Original-Signed-off-by: Jimmy Huang <jimmy.huang@mediatek.com> Original-Reviewed-on: https://chromium-review.googlesource.com/273994 Original-Reviewed-by: Julius Werner <jwerner@chromium.org> Original-Commit-Queue: Yidi Lin <yidi.lin@mediatek.com> Original-Tested-by: Yidi Lin <yidi.lin@mediatek.com> Reviewed-on: http://review.coreboot.org/10390 Tested-by: build bot (Jenkins) Reviewed-by: Furquan Shaikh <furquan@google.com> Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-06-02arm64: Guard prints in mmu.cFurquan Shaikh
We have observed issues with enabling CONFIG_SMP and adding prints before MMU is enabled on Tegra-based SoCs. This seems to be related to the hardware assisted locks and the restrictions laid down by ARMv8 spec. BUG=None BRANCH=None TEST=Boots to kernel prompt on smaug. Change-Id: I29a52f5a972baf396c01faba3ae3e5ecd27563e9 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: f52ee4b5b2e9b7f54eee0d105cb7e17f9a7e1613 Original-Change-Id: I432895560f468903c7beef00e78b6d38275a619c Original-Signed-off-by: Furquan Shaikh <furquan@google.com> Original-Reviewed-on: https://chromium-review.googlesource.com/272449 Original-Trybot-Ready: Furquan Shaikh <furquan@chromium.org> Original-Tested-by: Furquan Shaikh <furquan@chromium.org> Original-Reviewed-by: Furquan Shaikh <furquan@chromium.org> Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org> Reviewed-on: http://review.coreboot.org/10311 Tested-by: build bot (Jenkins) Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net> Reviewed-by: Furquan Shaikh <furquan@google.com> Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-06-02arm64: Decouple MMU functions from memrangesJulius Werner
The current arm64 MMU interface is difficult to use in pre-RAM environments. It is based on the memranges API which makes use of malloc(), and early stages usually don't have a heap. It is also built as a one-shot interface that requires all memory ranges to be laid out beforehand, which is a problem when existing areas need to change (e.g. after initializing DRAM). The long-term goal of this patch is to completely switch to a configure-as-you-go interface based on the mmu_config_range() function, similar to what ARM32 does. As a first step this feature is added side-by-side to the existing interface so that existing SoC implementations continue to work and can be slowly ported over one by one. Like the ARM32 version it does not garbage collect page tables that become unused, so repeated mapping at different granularities will exhaust the available table space (this is presumed to be a reasonable limitation for a firmware environment and keeps the code much simpler). Also do some cleanup, align comments between coreboot and libpayload for easier diffing, and change all error cases to assert()s. Right now the code just propagates error codes up the stack until it eventually reaches a function that doesn't check them anymore. MMU configuration errors (essentially just misaligned requests and running out of table space) should always be compile-time programming errors, so failing hard and fast seems like the best way to deal with them. BRANCH=None BUG=None TEST=Compile-tested rush_ryu. Booted on Oak and hacked MMU init to use mmu_config_range() insted of memranges. Confirmed that CRCs over all page tables before and after the change are equal. Change-Id: I93585b44a277c1d96d31ee9c3dd2522b5e10085b Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: f10fcba107aba1f3ea239471cb5a4f9239809539 Original-Change-Id: I6a2a11e3b94e6ae9e1553871f0cccd3b556b3e65 Original-Signed-off-by: Julius Werner <jwerner@chromium.org> Original-Reviewed-on: https://chromium-review.googlesource.com/271991 Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org> Reviewed-on: http://review.coreboot.org/10304 Tested-by: build bot (Jenkins) Reviewed-by: Paul Menzel <paulepanter@users.sourceforge.net> Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-04-27arch/arm64: update mmu translation table granule size, logic and macrosJimmy Huang
1. change mmu granule size from 64KB to 4KB 2. correct level 1 translation table creation logic 3. automatically calculate granule size related macros BRANCH=none BUG=none TEST=boot to kernel on oak board Change-Id: I9e99a3017033f6870b1735ac8faabb267c7be0a4 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: 2f18c4d5d9902f2830db82720c5543af270a7e3c Original-Change-Id: Ia27a414ab7578d70b00c36f9c063983397ba7927 Original-Signed-off-by: Jimmy Huang <jimmy.huang@mediatek.com> Original-Reviewed-on: https://chromium-review.googlesource.com/265603 Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org> Original-Commit-Queue: Yidi Lin <yidi.lin@mediatek.com> Original-Tested-by: Yidi Lin <yidi.lin@mediatek.com> Reviewed-on: http://review.coreboot.org/10009 Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org> Tested-by: build bot (Jenkins)
2015-04-22arm64: Correct shareability option for normal memoryFurquan Shaikh
In order to allow proper working of caches, set the correct shareability option for normal memory. BUG=chrome-os-partner:38222 BRANCH=None TEST=Compiles successfully for foster and SMP works. Change-Id: I5462cb0a2ff94a854f71f58709d7b2e8297ccc44 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: e092916780716ac80c3608c1bd8ca2901fbb3bd1 Original-Change-Id: Idd3c096a004d76a8fd75df2a884fcb97130d0006 Original-Signed-off-by: Furquan Shaikh <furquan@google.com> Original-Reviewed-on: https://chromium-review.googlesource.com/262992 Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org> Original-Tested-by: Furquan Shaikh <furquan@chromium.org> Original-Trybot-Ready: Furquan Shaikh <furquan@chromium.org> Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org> Reviewed-on: http://review.coreboot.org/9898 Tested-by: build bot (Jenkins) Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-03-28arm64: Reinit free_idx to 1 in mmu_initFurquan Shaikh
If mmu_init is called more than once then, free_idx should be reset to 1. Here, the assumption would be that mmu_init will not be called more than once. However, this is not necessarily true. Thus, free_idx should be reset to 1 every time we are initializing ttb from scratch. BUG=None BRANCH=None TEST=Compiles sucessfully and boots to kernel Change-Id: I5ac0af43346a492583380b0f15101390fc98d182 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: 398a68c3b08d82cfa521d235af2c1922629bdf56 Original-Change-Id: Idb7424df7dd577f263f12d1527dbd7fb89216d40 Original-Signed-off-by: Furquan Shaikh <furquan@google.com> Original-Reviewed-on: https://chromium-review.googlesource.com/216906 Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org> Original-Tested-by: Furquan Shaikh <furquan@chromium.org> Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org> Reviewed-on: http://review.coreboot.org/9068 Tested-by: build bot (Jenkins) Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-03-27arm64: remove printk() before console_init()Aaron Durbin
printk() shouldn't be called until the consoles have been initialized. This just so happened to work by luck. Once CONFIG_SMP is enabled that breaks because of spinlock usage in uncached memory. BUG=chrome-os-partner:31761 BRANCH=None TEST=Built with CONFIG_SMP and ramstage doesn't hang early. Change-Id: I54231db3c811c0d19c5c7fbaa406cacd1ff019ec Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: 31c3f972ac5c89472009b5b2cb7dbc0f02cfd9a0 Original-Change-Id: I6091b1e949e648b3435231946e5924260bf1807f Original-Signed-off-by: Aaron Durbin <adurbin@chromium.org> Original-Reviewed-on: https://chromium-review.googlesource.com/216920 Original-Reviewed-by: Furquan Shaikh <furquan@chromium.org> Reviewed-on: http://review.coreboot.org/9037 Tested-by: build bot (Jenkins) Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-03-27arm64: make mmu_enable() use previous ttb from mmu_init()Aaron Durbin
No need to pass in the same value for the ttb after just calling mmu_init(). All current users are setting this once and forgetting it. BUG=chrome-os-partner:31545 BRANCH=None TEST=Built and booted on ryu. Change-Id: Ie446d16eaf4ea65a34a9c76dd7c6c2f9b19c5d57 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: bd77461d483b513a569365673c83badc752f4aa8 Original-Change-Id: I54c7e4892d44ea6129429d8a46461d089dd8e2a9 Original-Signed-off-by: Aaron Durbin <adurbin@chromium.org> Original-Reviewed-on: https://chromium-review.googlesource.com/214772 Original-Reviewed-by: Furquan Shaikh <furquan@chromium.org> Reviewed-on: http://review.coreboot.org/9016 Tested-by: build bot (Jenkins) Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-03-26arm64: handle non-cacheable normal memoryAaron Durbin
Non-cacheable normal memory is needed when one wants an easy way to have a DMA region. That way all the reads and writes will be picked up by the CPU and the device without any cache management operations. BUG=chrome-os-partner:31293 BRANCH=None TEST=With a bevy of other patches can use a carved out DMA region for talking to USB. Change-Id: I8172f4b7510dee250aa561d040b27af3080764d7 Signed-off-by: Patrick Georgi <pgeorgi@chromium.org> Original-Commit-Id: a5bc7ab1709edd97d8795aa9687e6a0edf26ffc6 Original-Change-Id: I36b7fc276467fe3e9cec4d602652d6fa8098c133 Original-Signed-off-by: Aaron Durbin <adurbin@chromium.org> Original-Reviewed-on: https://chromium-review.googlesource.com/212160 Original-Reviewed-by: Furquan Shaikh <furquan@chromium.org> Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org> Reviewed-on: http://review.coreboot.org/8924 Tested-by: build bot (Jenkins) Reviewed-by: Stefan Reinauer <stefan.reinauer@coreboot.org>
2015-03-13armv8: Add mmu supportFurquan Shaikh
Add support for initializing and enabling mmu for armv8. Using 64KiB granule and 33 bits per VA, thus total VA address space is 6GiB. PA Range is 64GiB. Makes use of memrange library to get a list of all the mmap regions from the SoC to initialize XLAT table. Currently, all calculations in mmu.h are based on the assumptions that max 33 bits are used in VA and granule size is 64KiB. Changes in these assumptions will have to reflect in the dependent calculations as well. BUG=chrome-os-partner:30688 BRANCH=None TEST=Compiles rush successfully and boots until "payload not found". Goes past all the earlier alignment errors. Original-Change-Id: Iac1df15f0b81dcf64484a56b94f51357bcd67cc2 Original-Signed-off-by: Furquan Shaikh <furquan@google.com> Original-Reviewed-on: https://chromium-review.googlesource.com/208761 Original-Tested-by: Furquan Shaikh <furquan@chromium.org> Original-Reviewed-by: Aaron Durbin <adurbin@chromium.org> Original-Commit-Queue: Furquan Shaikh <furquan@chromium.org> (cherry picked from commit 6fe96360c03342115f849074f9e45a2c4e210705) Signed-off-by: Marc Jones <marc.jones@se-eng.com> Change-Id: I5360a3be95f198bd0b4f79b62f31228cc7a9c285 Reviewed-on: http://review.coreboot.org/8646 Reviewed-by: Furquan Shaikh <furquan@google.com> Tested-by: build bot (Jenkins)