Tracking issue for try_reserve: RFC 2116 fallible collection allocation

This is a tracking issue for the try_reserve part of the RFC "fallible collection allocation" (rust-lang/rfcs#2116).

Steps:

Implement the RFC Fallible allocation #48648
- Add HashSet::try_reserve: Replace HashMap implementation with SwissTable (as an external crate) #58623
Finalize the error type
Adjust documentation (see instructions on forge)
Stabilization PR (see instructions on forge)

API:

impl /* each of String, Vec<T>, VecDeque<T> */ {
    pub fn try_reserve(&mut self, additional: usize) -> Result<(), TryReserveError> {…}
    pub fn try_reserve_exact(&mut self, additional: usize) -> Result<(), TryReserveError> {…}
}

impl /* each of HashMap<K, V> and HashSet<T> */ {
    pub fn try_reserve(&mut self, additional: usize) -> Result<(), TryReserveError> {…}
}

/// The error type for `try_reserve` methods.
#[derive(Clone, PartialEq, Eq, Debug)]
pub enum TryReserveError { // in std::collections
    /// Error due to the computed capacity exceeding the collection's maximum
    /// (usually `isize::MAX` bytes).
    CapacityOverflow,

    /// The memory allocator returned an error
    AllocError {
        /// The layout of allocation request that failed
        layout: Layout,

        #[doc(hidden)]
        #[unstable(feature = "container_error_extra", issue = "0", reason = "\
            Enable exposing the allocator’s custom error value \
            if an associated type is added in the future: \
            https://github.com/rust-lang/wg-allocators/issues/23")]
        non_exhaustive: (),
    },
}

impl From<LayoutErr> for TryReserveError {
    fn from(_: LayoutErr) -> Self {
        TryReserveError::CapacityOverflow
    }
}

Hi, this feature will require oom and try_reserve to be implemented. I'm needing try_reserve and I'm willing to work (slowly) on it if noone else does.
I see that the previous commit by @gankro is here and seems to match the RFC description.
Is there anything else I should consider before rebasing that PR?

I can use some mentorship on what's the best way to implement oom=panic/abort . Should I emit a global variable through rustc depending on this case?

I was taking inspiration from #32900 but there might be an easier way to emit a boolean global variable.

cc @gankro, can you help mentor?

try_reserve methods are implemented in Nightly, but the accepted RFC also includes the ability to make allocation failure cause a panic rather than abort and that part is not implemented. Unfortunately the RFC does not discuss how this would be implemented, other than being controlled by a flag in Cargo.toml. It’s not clear to me how this can work.

Unfortunately the RFC does not discuss how this would be implemented, other than being controlled by a flag in Cargo.toml. It’s not clear to me how this can work.

We've been talking about panic::set_hook-like hooking for the new oom lang item in #49668, this could be how this works.

@alexcrichton

@alexcrichton, how does #51041 fit with the goal of implementing some way to opt into oom=panic?

It's super minor but TryReserveErrorKind is missing periods at the end of sentences in the documentation.

It's super minor but TryReserveErrorKind is missing periods at the end of sentences in the documentation.

The only such sentence I see there is "The memory allocator returned an error"; the others are just sentence fragments, which may not deserve periods (for example, where sentence fragments may be used in Wikipedia, their style guide (probably the result of compromise between many strong opinions) says these "should in most cases not end with a period").

the others are just sentence fragments, which may not deserve periods

https://doc.rust-lang.org/1.0.0/style/style/comments.html#sentence-structure

All doc comments, including the summary line, should begin with a capital letter and end with a period, question mark, or exclamation point. Prefer full sentences to fragments.

Sentence fragments should end with a period in Rust, even if full sentences are preferred.

Important news! HashMap got method try_insert ( #82766 ), where try_ means something completely different to Box::try_new. Please, prevent somehow this try_insert from becoming stable

Please file an issue dedicated to the naming of try_insert since this thread is already covering other topics.

The comment of example code is seems wrong.
If size of data is about 60% of system memory, try_reserve might success because OS doesn't allocate physical memory immediately.
Writing to allocated memory causes page fault and OS allocates memory.
Sum of size of old data and new data is about 120% of system memory.
So, OOM might occur.

List of the comments

$ rg -C3 "can't OOM"
library/std/src/ffi/os_str.rs
311-    ///     // Pre-reserve the memory, exiting if we can't
312-    ///     s.try_reserve(OsStr::new(data).len())?;
313-    ///
314:    ///     // Now we know this can't OOM in the middle of our complex work
315-    ///     s.push(data);
316-    ///
317-    ///     Ok(s)
--
382-    ///     // Pre-reserve the memory, exiting if we can't
383-    ///     s.try_reserve_exact(OsStr::new(data).len())?;
384-    ///
385:    ///     // Now we know this can't OOM in the middle of our complex work
386-    ///     s.push(data);
387-    ///
388-    ///     Ok(s)

library/alloc/src/string.rs
1098-    ///     // Pre-reserve the memory, exiting if we can't
1099-    ///     output.try_reserve(data.len())?;
1100-    ///
1101:    ///     // Now we know this can't OOM in the middle of our complex work
1102-    ///     output.push_str(data);
1103-    ///
1104-    ///     Ok(output)
--
1139-    ///     // Pre-reserve the memory, exiting if we can't
1140-    ///     output.try_reserve_exact(data.len())?;
1141-    ///
1142:    ///     // Now we know this can't OOM in the middle of our complex work
1143-    ///     output.push_str(data);
1144-    ///
1145-    ///     Ok(output)

library/alloc/src/collections/binary_heap.rs
994-    ///     // Pre-reserve the memory, exiting if we can't
995-    ///     heap.try_reserve_exact(data.len())?;
996-    ///
997:    ///     // Now we know this can't OOM in the middle of our complex work
998-    ///     heap.extend(data.iter());
999-    ///
1000-    ///     Ok(heap.pop())
--
1029-    ///     // Pre-reserve the memory, exiting if we can't
1030-    ///     heap.try_reserve(data.len())?;
1031-    ///
1032:    ///     // Now we know this can't OOM in the middle of our complex work
1033-    ///     heap.extend(data.iter());
1034-    ///
1035-    ///     Ok(heap.pop())

library/alloc/src/collections/vec_deque/mod.rs
776-    ///     // Pre-reserve the memory, exiting if we can't
777-    ///     output.try_reserve_exact(data.len())?;
778-    ///
779:    ///     // Now we know this can't OOM(Out-Of-Memory) in the middle of our complex work
780-    ///     output.extend(data.iter().map(|&val| {
781-    ///         val * 2 + 5 // very complicated
782-    ///     }));
--
813-    ///     // Pre-reserve the memory, exiting if we can't
814-    ///     output.try_reserve(data.len())?;
815-    ///
816:    ///     // Now we know this can't OOM in the middle of our complex work
817-    ///     output.extend(data.iter().map(|&val| {
818-    ///         val * 2 + 5 // very complicated
819-    ///     }));

library/alloc/src/vec/mod.rs
893-    ///     // Pre-reserve the memory, exiting if we can't
894-    ///     output.try_reserve(data.len())?;
895-    ///
896:    ///     // Now we know this can't OOM in the middle of our complex work
897-    ///     output.extend(data.iter().map(|&val| {
898-    ///         val * 2 + 5 // very complicated
899-    ///     }));
--
936-    ///     // Pre-reserve the memory, exiting if we can't
937-    ///     output.try_reserve_exact(data.len())?;
938-    ///
939:    ///     // Now we know this can't OOM in the middle of our complex work
940-    ///     output.extend(data.iter().map(|&val| {
941-    ///         val * 2 + 5 // very complicated
942-    ///     }));

@Toru3

@Toru3 Overcommit is a known limitation, and it's been considered in the RFC. You can mitigate the problem on the OS level either by disabling overcommit, or running code in containers with limited amount of memory. Within Rust programs you can reduce chance of hitting the problem by using the cap allocator with a limit below the amount of physical memory available.

Would it be possible/wise to put TryReserveError into core ? I may need to have access to this struct when my crate compile without alloc.

Added try_with_capacity

Just to check my understanding, are TryReserveError::kind and TryReserveErrorKind the only blockers now?

Some concerns about the contents of TryReserveErrorKind rust-lang/wg-allocators#106 (comment)

test(testing/oom): pin RawVec capacity-guard for synthetic OOM helper (…

Navigation Menu

Repository navigation

Tracking issue for `try_reserve`: RFC 2116 fallible collection allocation #48043

126 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Tracking issue for try_reserve: RFC 2116 fallible collection allocation #48043

Description

Activity

snf commented on Feb 22, 2018

snf commented on Mar 3, 2018

aturon commented on Mar 5, 2018

SimonSapin commented on Apr 5, 2018

glandium commented on May 4, 2018

SimonSapin commented on May 29, 2018

126 remaining items

orlp commented on Dec 27, 2021

8573 commented on Dec 27, 2021

orlp commented on Dec 27, 2021

safinaskar commented on Jan 21, 2022

SimonSapin commented on Jan 24, 2022

Toru3 commented on Aug 16, 2022

kornelski commented on Aug 25, 2022

Stargateur commented on Mar 13, 2024

studyingegret commented on Jan 2, 2026

abrahem79 commented on Feb 9, 2026

tgross35 commented on Mar 31, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Tracking issue for `try_reserve`: RFC 2116 fallible collection allocation #48043