Commit graph

169 commits

Author SHA1 Message Date
Daniel Micay
3bee8d3e0e fix realloc from small sized allocations with above PAGE_SIZE alignment
Large allocations don't always have a size larger than the maximum slab
size class because alignment larger than PAGE_SIZE is handled via large
allocations. The general case in realloc was assuming small sizes imply
slab allocations which isn't guaranteed.

Alignment above PAGE_SIZE is rare and realloc doesn't preserve alignment
so passing aligned allocations to realloc is also rare. In practice, it
ends up doing invalid accesses within the reserved metadata region which
will almost always crash due to it being largely PROT_NONE memory and it
having an extremely high likelihood of indexing into the PROT_NONE areas
rather than the actual metadata. That means if this impacted an app, it
would currently be crashing in practice. Due to the reserved region for
metadata and the fact that it would be crashing, this can be ruled out
as a security concern but is potentially an extremely rare compatibility
issue if there's any code using this.

Reported-by: Stefan Rus <stefan@photonspark.com>
2026-02-22 14:58:24 -05:00
Daniel Micay
e03579253a preserve PROT_MTE when releasing memory 2024-10-12 03:19:16 -04:00
Daniel Micay
9739cb4690 use wrapper for calling memory_map_mte 2024-10-12 03:19:03 -04:00
Daniel Micay
6402e2b0d4 reduce probability hint for is_memtag_enabled 2024-10-12 03:17:44 -04:00
Dmitry Muhomor
7268189933 mte: use tag 0 for freed slots, stop reserving tag 15 2024-01-23 12:56:54 -05:00
jvoisin
61821b02c8 Clarify a bit why a particular magic number was chosen 2023-11-16 14:25:54 -05:00
Dmitry Muhomor
72dc236d5f mte: add untag_pointer() variant for const pointers 2023-10-30 14:20:53 -04:00
Dmitry Muhomor
be08eeee2d mte: update comment about skipped tag array update in deallocate_small() 2023-10-30 14:20:53 -04:00
Dmitry Muhomor
c75cb4c3f3 mte: refactor tag_and_clear_slab_slot()
Explicitly call is_memtag_enabled() before calling tag_and_clear_slab_slot() to make it clearer that
memory is not zeroed when MTE is disabled.
2023-10-30 14:20:53 -04:00
Dmitry Muhomor
b560431c01 mte: note why 0 tag is excluded 2023-10-30 14:20:53 -04:00
Dmitry Muhomor
03883eb2ce mte: rename arm_mte_store_tags_and_clear() to arm_mte_tag_and_clear_mem() 2023-10-30 14:20:53 -04:00
Dmitry Muhomor
7a6dbd8152 mte: add comment about the reserved slab canary value 2023-10-30 14:20:53 -04:00
Dmitry Muhomor
93aa9eefe4 mte: make h_malloc_disable_memory_tagging() thread-safe 2023-10-30 14:20:53 -04:00
Dmitry Muhomor
01a199e19e mte: move is_memtag_enabled to read-only allocator data 2023-10-30 14:20:53 -04:00
Dmitry Muhomor
f042a6b9b0 android: add function for disabling MTE at runtime
On Android, MTE is always enabled in Zygote, and is disabled after fork for apps that didn't opt-in
to MTE.

Depends on the slab canary adjustments in the previous commit.
2023-10-30 14:20:53 -04:00
Dmitry Muhomor
001fc86585 mte: disable slab canaries when MTE is on
Canary with the "0" value is now reserved to support re-enabling slab canaries if MTE is turned off
at runtime.
2023-10-30 14:20:53 -04:00
Dmitry Muhomor
70c91f4c3e mte: disable write-after-free check for slab allocations when MTE is on
Freed slab memory is tagged with a reserved tag value that is never used for live allocations.
2023-10-30 14:20:53 -04:00
Dmitry Muhomor
e3686ae457 add support for Arm MTE memory tagging
- tag slab allocations with [1..14] tags
- tag freed slab allocations with the "15" tag value to detect accesses to freed slab memory
- when generating tag value for a slab slot, always exclude most recent tag value for that slot
(to make use-after-free detection more reliable) and most recent tag values of its immediate
neighbors (to detect linear overflows and underflows)
2023-10-30 14:20:53 -04:00
Christian Göttsche
9cb4e6daf6 do not skip init() in realloc()
If N_ARENA is greater than 1 `thread_arena` is initially to N_ARENA,
which is an invalid index into `ro.size_class_metadata[]`.

The actual used arena is computed in init().

Ensure init() is called if a new thread is only using realloc() to avoid
UB, e.g. pthread_mutex_lock() might crash due the memory not holding an
initialized mutex.

Affects mesa 23.2.0~rc4.

Example back trace using glmark2 (note `arena=4` with the default
N_ARENA being 4):

    Program terminated with signal SIGSEGV, Segmentation fault.
    #0  ___pthread_mutex_lock (mutex=0x7edff8d3f200) at ./nptl/pthread_mutex_lock.c:80
            type = <optimized out>
            __PRETTY_FUNCTION__ = "___pthread_mutex_lock"
            id = <optimized out>
    #1  0x00007f0ab62091a6 in mutex_lock (m=0x7edff8d3f200) at ./mutex.h:21
    No locals.
    #2  0x00007f0ab620c9b5 in allocate_small (arena=4, requested_size=24) at h_malloc.c:517
            info = {size = 32, class = 2}
            size = 32
            c = 0x7edff8d3f200
            slots = 128
            slab_size = 4096
            metadata = 0x0
            slot = 0
            slab = 0x0
            p = 0x0
    #3  0x00007f0ab6209809 in allocate (arena=4, size=24) at h_malloc.c:1252
    No locals.
    #4  0x00007f0ab6208e26 in realloc (old=0x72b138199120, size=24) at h_malloc.c:1499
            vma_merging_reliable = false
            old_size = 16
            new = 0x0
            copy_size = 139683981990973
    #5  0x00007299f919e556 in attach_shader (ctx=0x7299e9ef9000, shProg=0x7370c9277d30, sh=0x7370c9278230) at ../src/mesa/main/shaderapi.c:336
            n = 1
    #6  0x00007299f904223e in _mesa_unmarshal_AttachShader (ctx=<optimized out>, cmd=<optimized out>) at src/mapi/glapi/gen/marshal_generated2.c:1539
            program = <optimized out>
            shader = <optimized out>
            cmd_size = 2
    #7  0x00007299f8f2e3b2 in glthread_unmarshal_batch (job=job@entry=0x7299e9ef9168, gdata=gdata@entry=0x0, thread_index=thread_index@entry=0) at ../src/mesa/main/glthread.c:139
            cmd = 0x7299e9ef9180
            batch = 0x7299e9ef9168
            ctx = 0x7299e9ef9000
            pos = 0
            used = 3
            buffer = 0x7299e9ef9180
            shared = <optimized out>
            lock_mutexes = <optimized out>
            batch_index = <optimized out>
    #8  0x00007299f8ecc2d9 in util_queue_thread_func (input=input@entry=0x72c1160e5580) at ../src/util/u_queue.c:309
            job = {job = 0x7299e9ef9168, global_data = 0x0, job_size = 0, fence = 0x7299e9ef9168, execute = <optimized out>, cleanup = <optimized out>}
            queue = 0x7299e9ef9058
            thread_index = 0
    #9  0x00007299f8f1bcbb in impl_thrd_routine (p=<optimized out>) at ../src/c11/impl/threads_posix.c:67
            pack = {func = 0x7299f8ecc190 <util_queue_thread_func>, arg = 0x72c1160e5580}
    #10 0x00007f0ab5aa63ec in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:444
            ret = <optimized out>
            pd = <optimized out>
            out = <optimized out>
            unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139683974242608, 2767510063778797177, -168, 11, 140727286820160, 126005371879424, -4369625917767903623, -2847048016936659335}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0,
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
            not_first_call = <optimized out>
    #11 0x00007f0ab5b26a2c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
2023-09-26 20:03:02 -04:00
Christian Göttsche
7d75acc62a use strict prototype
h_malloc.c:83:21: error: function declaration isn’t a prototype [-Werror=strict-prototypes]
       83 | static inline void *get_slab_region_end() {
          |                     ^~~~~~~~~~~~~~~~~~~
2023-06-10 14:18:27 -04:00
Daniel Micay
64dad0a69f drop legacy glibc support 2023-06-10 14:04:46 -04:00
Daniel Micay
d3152b8e8f preserve errno for free calls
This is a future POSIX requirement recently implemented by musl and
glibc.
2023-02-17 13:07:26 -05:00
Daniel Micay
2e9daf3122 merge fprintf/fputs calls in malloc_info 2023-02-17 13:07:26 -05:00
Daniel Micay
2250130c53 remove unnecessary UNUSED marker 2022-09-16 01:03:47 -04:00
Daniel Micay
8f38bbdee6 add configuration for self-init
This needs to be disabled for compatibility with the exploit protection
compatibility mode on GrapheneOS. hardened_malloc shouldn't be trying to
initialize itself when exploit protection compatibility mode is enabled.
This has to be handled in our Bionic integration instead.
2022-09-14 03:41:31 -04:00
jvoisin
b511696c55 clean up includes and remove non-portable includes
This marginally increases the portability of hardened_malloc,
eg. on OSX.
2022-02-07 07:14:51 -05:00
Daniel Micay
448170a412 fix case for non-macro constant 2022-01-21 23:59:37 -05:00
Daniel Micay
995ce07d45 add is_init likely/unlikely markers 2022-01-21 19:46:49 -05:00
Daniel Micay
c9d1abcd7e explicitly mark fatal error conditions unlikely 2022-01-21 19:45:05 -05:00
Daniel Micay
8f0b252c33 mark more out-of-memory conditions as unlikely 2022-01-21 19:03:02 -05:00
Daniel Micay
3cffc1e1af treat zero size malloc as unlikely
Calls to malloc with a zero size are extremely rare relative to normal
usage of the API. It's generally only done by inefficient C code with
open coded dynamic array implementations where they aren't handling zero
size as a special case for their usage of malloc/realloc. Efficient code
wouldn't be making these allocations. It doesn't make sense to optimize
for the performance of rare edge cases caused by inefficient code.
2022-01-21 18:27:04 -05:00
Daniel Micay
b3d78bd5f6 use static const for local constants 2022-01-16 21:02:17 -05:00
Daniel Micay
8d61e63274 add comment about special small size classes 2022-01-16 20:50:49 -05:00
Daniel Micay
81cf2f27a0 calculate slab size class instead of array loop 2022-01-16 16:18:14 -05:00
Daniel Micay
d8cb2d9f7a use consistent wrappers around clz/ffs 2022-01-16 15:39:59 -05:00
Daniel Micay
86f9c739ee define constant for u64 bit width 2022-01-16 15:06:36 -05:00
Daniel Micay
536f852538 reuse a single size alignment implementation 2022-01-16 14:44:28 -05:00
Daniel Micay
2a5662948e rename bitmap manipulation functions 2022-01-04 12:14:55 -05:00
Daniel Micay
d1c39edc9b use const for malloc_object_size API 2022-01-04 10:14:41 -05:00
Daniel Micay
5f32942263 get rid of canary_value when canaries are disabled 2022-01-03 20:39:30 -05:00
Daniel Micay
3696f071a4 use SLAB_CANARY for conditional checks 2022-01-03 02:17:04 -05:00
Daniel Micay
8ae78237ae avoid unnecessarily mixing 32-bit and 64-bit ints
It's ever so slightly faster to stick to stick to 64-bit arithmetic and
it avoids clang tidy being unhappy about the implicit widening.
2022-01-03 00:54:43 -05:00
Daniel Micay
3f8e9d3184 make MREMAP_MOVE_THRESHOLD into size_t constant
This avoids a clang-tidy warning and is a bit cleaner.
2022-01-03 00:32:06 -05:00
jvoisin
9142a9376b Add a bunch of const qualifiers 2021-12-30 21:25:16 -05:00
jvoisin
0655c1d024 Add a missing const 2021-12-26 18:19:59 -05:00
Daniel Micay
e41d37c3de remove unnecessary else 2021-09-30 10:57:05 -04:00
Daniel Micay
be6dde66f9 fix missing include for Intel MPK support 2021-05-21 09:07:28 -04:00
Daniel Micay
27fcfccb67 make __GLIBC_PREREQ check for mallinfo2 portable 2021-05-12 22:53:20 -04:00
Daniel Micay
da190f1469 mark pvalloc error path as unlikely 2021-05-12 21:01:13 -04:00
Daniel Micay
b0f81365a8 reuse code for aligned allocation API entry points 2021-05-12 20:59:04 -04:00