On 8/13/22 08:04, Kevin Kofler via devel wrote: CCing Daniel Micay who wrote hardened_malloc. -- Sincerely, Demi Marie Obenour (she/her/hers)

On Mon, Aug 15, 2022 at 07:39:46PM -0700, John Reiser wrote: It has far less fragmentation than glibc malloc. It also has far lower metadata overhead since there are no headers on allocations and only a few bits consumed per small allocation. glibc has over 100% metadata overhead for 16 byte allocations while for hardened_malloc it's a very low percentage. Of course, you need to compare with slab allocation quarantines and slab allocation canaries disabled in hardened_malloc. Internal malloc means fragmentation caused by size class rounding. There is no way to have size classes that aren't multiples of 16 due to it being required by the x86_64 and arm64 ABI. glibc has over 100% overhead for 16 byte allocations due to header metadata and other metadata. It definitely isn't lighter for those compared to a modern slab allocator. There's a 16 byte alignment requirement for malloc on x86_64 and arm64 so there's no way to have any size classes between the initial multiples of 16. Slab allocation canaries are an optional hardened_malloc feature adding 8 byte random canaries to the end of allocations, which in many cases will increase the size class if there isn't room within the padding. Slab allocation quarantines are another optional feature which require dedicating substantial memory to avoiding reuse of allocations. You should compare without the optional features enabled as a baseline because glibc doesn't have any of those security features, and the baseline hardened_malloc design is far more secure. malloc(0) is not implemented as malloc(1) in hardened_malloc and does not use any memory for the data, only the metadata, which is a small percentage of the allocation size even for 16 byte allocations since there is only slab metadata for the entire slab and bitmaps to track which slots are used. There are no allocation headers. Doing hundreds of thousands of malloc(0) allocations only uses a few bytes of memory in hardened_malloc. Each allocation requires a bit in the bitmap and each slab of 256x 16 byte allocations (4096 byte slab) has slab metadata. All the metadata is in a dedicated metadata region. I strong recommend reading all the documentation thoroughly: https://github.com/GrapheneOS/hardened_malloc/blob/main/README.md hardened_malloc is oriented towards security and provides a bunch of important security properties unavailable with glibc malloc. It also has lower fragmentation and with the optional security features disabled also lower memory usage for large processes and especially over time. If you enable the slab quarantines, that's going to use a lot of memory. If you enable slab canaries, you give up some of the memory usage reduction from not having per-allocation metadata headers. Neither of those features exists in glibc malloc, jemalloc, etc. so it's not really fair to enable the optional security features for hardened_malloc and compare with allocators without them. Slab allocation quarantines in particular inherently require a ton of memory in order to delay reuse of allocations for as long of a time as is feasible. This pairs well with zero-on-free + write-after-free-check based on zero-on-free, since if any non-zero write occurs while quarantined/freed it will be detected before the allocation is reused. As long as zero-on-free is enabled, which it is even for the sample light configuration, then all memory is known to be zeroed at allocation time, which is how the write-after-free-check works. All of these things are supplementary optional features, NOT the core security features. The core security features are the baseline design of not having any inline metadata, having entirely separate fully statically reserved address space regions each with their own high entropy random base for all allocation metadata (1 region) and each size class (each has a separate region), absolutely never reusing address space between the regions, etc. It provides very substantial security benefits over a completely non-hardened allocator with a legacy easy to exploit design such as glibc malloc and only a few mostly non-working sanity checks. There are other approaches which take a middle ground, but hardened_malloc is focused on security first, with very low fragmentation (dramatically lower than glibc) and also lower memory usage for large processes when slab allocation quarantines are disabled and especially when slab canaries are also disabled. Try using hardened_malloc for something like the Matrix Synapse server with the light configuration (slab allocation quarantine not used) and compare to glibc malloc. It uses far less memory and unlike glibc malloc doesn't end up 'leaking' tons of memory over time from fragmentation. Disable slab canaries and try again, it will be even lower, although probably not particularly noticeably. If you choose to use the very memory expensive slab quarantine feature which is not enabled in the standard light configuration, that's your choice. Also, you hardened_malloc doesn't use a thread cache for security reasons. It invalidates many of the security properties. If you compare to glibc malloc in the light configuration with tcache disabled in glibc malloc it will compare well, and hardened_malloc can scale better when given enough arenas. If you want to make the substantial security sacrifices required for a traditional thread cache, then I don't think hardened_malloc makes sense, which is why it doesn't include the option to do thread caching even though it'd be easy to implement. It may one day include the option to do thread batched allocation, but it isn't feasible to do it for deallocation without losing a ton of the strong security properties.

On Sat, Aug 27, 2022 at 9:14 AM Carlos O'Donell <carlos(a)redhat.com&gt; wrote: A third option (or maybe it's an improvement to the default?), since the choice of allocators seems to come up consistently, could be to consider seriously (and is likely not a trivial project) the idea of making it easier to switch allocators. However to echo what Timothée said, there's value in packaging hardened_malloc for Fedora to make it available to users. It's much too early to start talking about switching defaults IMO. Sid

I think that the first steps here would be to: - package it in Fedora - write a documentation page on how to use it (the quick docs may be a good place: https://docs.fedoraproject.org/en-US/quick-docs/) - do a lot of testing and benchmarks to get memory and performance numbers for each major Fedora use case (workstation, server, IoT, etc.)

Adding Daniel for awareness. Regards. Pablo El mié., 31 ago. 2022 16:09, John Reiser <jreiser(a)bitwagon.com&gt; escribió:

On 9/5/22 21:02, Daniel Micay via devel wrote: The default configuration was used. " ...; make" produces out/libhardened_malloc.so and no other shared library. The use case is a builder and distributor of software packages to a large, diverse audience. There is concern about the possibility of malware attacking the build process, a "supply-chain attack". Of course there are other protections already in place, but the possibility of better protection is reasonable to investigate. A network search revealed a dearth of end-to-end performance measurements, and/or comparisons based on actual data. I reported an actual measurement and comparison of two allocators using commonly-available tools and a documented, repeatable methodology. The choice of which two allocators is reasonable for the use case. > [[snip]]

On Wed, Aug 31, 2022 at 10:19:51AM -0700, John Reiser wrote: The comparison is being done incorrectly. Since hardened_malloc builds both a lightweight and heavyweight library by default, and since I already explained this and that the lightweight library still has optional security features enabled, it doesn't seem to have been done in good faith. My previous posts where I provided both concise and detailed information explaining differences and the approach were ignored. Why is that? As I said previously, hardened_malloc has a baseline very hardened allocator design. It also has entirely optional, expensive security features layered on top of that. I explained in detail that some of those features have a memory cost. Slab allocation canaries have a small memory cost and slab allocation quarantines have a very large memory cost especially with the default configuration. Those expensive optional features each have an added performance cost too. Measuring with 100% of the expensive optional features enabled and trying to portray the performance of the allocator solely based on that is simply incredibly misleading and disregards all of my previous posts in the thread. hardened_malloc builds both a lightweight and heavyweight library by default. The lightweight library still has 2 of the optional security features enabled. None of the optional security features is provided by glibc malloc and if you want to compare the baseline performance then none of those should be enabled for a baseline comparison. Take the light configuration, disable slab allocation canaries and full zero-on-free, and there you go. I also previously explained that hardened_malloc does not include a thread cache for security reasons inherent to the concept of a thread cache. An array-based thread cache with out-of-line metadata would still hurt security, but would be a more suitable approach than a free list compromising the otherwise complete lack of inline metadata. Compare hardened_malloc with the optional security features disabled to glibc malloc and also to glibc malloc with tcache disabled. It's easy enough to stick a thread cache onto hardened_malloc and if there was demand for that I could implement it in half an hour. At the moment, the current users of hardened_malloc don't want to make the sacrifice of losing 100% reliable detection of invalid frees along with the many other benefits lost by doing that.

On 9/5/22 19:45, Daniel Micay wrote: That claim is false. The Makefile for commit 72fb3576f568481a03076c62df37984f96bfdfeb on Tue Aug 16 07:47:26 2022 -0400 (which is the HEAD of the trunk) begins ===== VARIANT := default ifneq ($(VARIANT),) CONFIG_FILE := config/$(VARIANT).mk include config/$(VARIANT).mk endif ifeq ($(VARIANT),default) SUFFIX := else SUFFIX := -$(VARIANT) endif OUT := out$(SUFFIX) ===== and builds only one library, namely $OUT/libhardened_malloc$SUFFIX.so which for the case of "no options specified" is out/libhardened_malloc.so . If would be better for external perception if the name "libhardened_malloc.so" were changed to something like "libhardened_malloc-strong.so". Having both -strong and -light versions built every time would highlight the difference, and force the user to decide, and encourage the analysis that is required to make an informed choice. and since I I measured the result of building and using with the default options. Unpack the source, use "as-is" with no adjustment, no tweaking, no tuning. If the default source is not appropriate to use as widely as implied by the name "malloc" (with no prefix and no suffix on the subroutine name), then the package is not suitable for general use. Say so immediately at the beginning of the README.md: "This software is not suitable for widespread general use, unless adjusted according to the actual use cases." I reported an end-to-end measurement and comparison based on data. Where have you reported actual end-to-end measurements and comparisons? > [[snip]]