Repository navigation
Tracking Issue for thin_box #92791
Description
Activity
Example PR using ThinBox as a substitute for
Box<dyn Error + ...>: https://github.com/yaahc/pgx/pull/1/files- added a commit that references this issue
on Apr 9, 2022 One unfortunate thing about ThinBox is that it's invariant rather than covariant, like Box. For example, this playground fails to compile: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=f6348153d6555716ce86926ac1928a4e.
This happens because of the
<T as Pointee>::Metadataprojection that is used internally. Projections like these currently require invariance. I'm not sure if there's a way to fix it, but it's unfortunate to differ from box in this way IMO. Arguably, this is more of an issue with the Pointee trait than with ThinBox itself, though, since it's not like ThinBox can fix it -- it also is less likely to matter in most of ThinBox's proposed use cases (although it would be nice if the version from the stdlib could somehow do better here).A use case that's slightly more realistic for ThinBox is https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=e2659853940505c62b73b21deb079195:
#![feature(thin_box)] use std::boxed::ThinBox; pub trait Foo {} pub fn add_foo(foos: &mut Vec<ThinBox<dyn Foo + '_>>, foo: ThinBox<dyn Foo + 'static>) { foos.push(foo); }
This fails with a lifetime error, because
'_is not'static. This is actually safe usage, though, and withBoxit works fine: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=dc3e919e611f4e91ac0567db47ac4d92.FWIW, I also noted the invariance of
<T as Pointee>::Metadatain #81513 (comment).Reacted by Thom ChiovoloniSendandSyncare also missing, compared toBox<T>, becauseNonNull<_>blocks them by default. I think it should be perfectly fine to justify manual impls forThinBox<T>though.Reacted by Jane Losare-LusbyOh, it should definitely be Send and Sync.
Is it not supposed to implement Eq? because Box does.
I was wondering, Because I get this error:
the trait Eq is not implemented for ThinBox<Vec<Item>>Eqis not implemented yet, but it seems reasonable to add, and eventually mirror most ofBox's API. For now, I think the initial implementation was intentionally kept light.Reacted by Jane Losare-Lusby16 remaining items
What is the difference between this and a normal
Box?Reacted by Nathan Vander WiltWhat is the difference between this and a normal
Box?A
Boxwill store unsized types (slices,str, object traits, etc.) as a pointer to the data and some metadata related to the type (slices andstrs store their length, and object traits store a pointer to a table with the function pointers to the trait's methods).A
ThinBoxwould always only store a single pointer on the stack, by writing the metadata that a regularBoxwould have on the stack alongside the data it points to.This would make it easier to pass pointers to unsized types through FFI boundries.
Example of
Box<str>v.ThinBox<str>
Box<str>
[ Metadata ][ Pointer to string data ]ThinBox<str>
[ Pointer to data ] --> [ Metadata, string data ]Reacted by NoahJelen, Claudia Meadows, Kisaragi, tigregalis, Joseph C. Sible, Nathan Vander Wilt, Lynne and Mavity the MadityIs it guaranteed what the layout of Metadata will be? Would it be feasible to make a ThinBox FFI-safe?
If we can guarantee:
- the final item in
Metadatais ausizecontaining the length - length is immediately next to the first
T - a
ThinBox<[T]>isrepr(transparent)over a*const () - the pointer contained inside a
ThinBox<[T]>points to the firstT
a
ThinBox<[T]>is safe to pass through FFI, and the FFI users can just offset backwards by ausizeto get a pointer to the length. however, they couldn't free theThinBox<[T]>, as they'd need to offset to the beginning of the entire buffer, wheremallocwould store the size of the buffer to free later on, unless, of course,Metadatahas a guaranteed size and (semi-optionally) layout.- the final item in
If we can guarantee:
- the final item in
Metadatais ausizecontaining the length
I was under the impression this is exactly the kind of thing we didn't want to have, since we want a path forward for types which are unsized but not length-prefixed, like
CStr, and we don't want to guarantee exactly where the size field is stored in stuff like vtables.Reacted by Kornel- the final item in
In that case, perhaps it could be made optional somehow - the problem there would be how we make the length optional. It's valid to have a slice with a length of 0, and a pointer that's not NULL. You can also make a
Box<[T]>from aVec<T>with a length of 0. If we allocate an additional byte for a "does this type have an immediate length?", we can reference that instead.In that case, perhaps it could be made optional somehow - the problem there would be how we make the length optional. It's valid to have a slice with a length of 0, and a pointer that's not NULL. You can also make a
Box<[T]>from aVec<T>with a length of 0. If we allocate an additional byte for a "does this type have an immediate length?", we can reference that instead.Worth noting that, for fat empty slices, the slice pointer is in practice still never null, to ensure that
Some(&[])andNonecan be told apart without needing an extra discriminant byte. Instead, the empty slice pointer is usuallyNonNull::<T>::dangling(), but not always, and can be just about any nom-zero value.Not that you should ever depend on this for correctness, mind you. This is just the current implementation to my knowledge.
Reacted by KornelYou can depend on it being non-zero because
&Tis guaranteed to non-zero, but another reason why you can't depend on an empty slice being equivalent todanglingis because you can easily create an empty slice from a non-empty slice, e.g.s[..0]. This is also what you would get from callingas_sliceon an exhaustedslice::Iter; the empty slice's pointer would be somewhere between the start and past-end of the valid slice, depending on the order of iteration.Empty boxes might be able to optimise differently since you generally don't want to reference a valid allocation if it's empty, although there are also other cases that depend on pointer-equality being different even for empty allocations.
I would just generally not rely on pointer checking to determine size. The main case of a no-metadata DST is CStr, which is never an empty allocation; an empty string is
[0]. Even if we statically allocate one "empty string" that we reuse, that would still be a valid pointer to a valid buffer, just the same one.Reacted by Kornel- added a commit that references this issue
on Feb 12, 2024 More ways of making ThinBox slices would likely be helpful - e.g.:
- a
FromIteratorimplementation - a way of copying or cloning an existing slice
- a
ThinBox::new_uninit_slice(n: usize)for only allocating the slice
- a
Now that I'm thinking about this more, I'm starting to like the basic idea in #92791 (comment) more and more. Here's my intuition:
- Unsizing types to fat pointers involves pairing a reference to the original value with generated metadata for the final field.
- Unsizing types to thin pointers involves storing the generated metadata before the type, and then returning a pointer at the metadata's offset. It's different from the default, and the type system currently cannot represent changing unsizing placement like that.
- In both cases, it's separated from allocation, and heap allocators do not need any knowledge of whether the pointer is thin or fat, only that it's a pointer.
I do feel it can't be fully done in userland, though, and it feels more like a
dyn Traitthan aPin<P>in the way it modifies unsizing, rather than merely decorating it. Makes me feel that&thin T,Box<thin T>, and so on would provide a better, cleaner interface. For a thin trait object, you'd do&thin dyn Trait, and for a thin slice reference, it'd be&thin [Foo]. The compiler could just reserve that storage on the stack transparently if a&thin Tis requested. And for allocators,std::mem::size_of_val(&thin value)returns the metadata size plusstd::mem::size_of_val(&value).Reacted by kennytm, Sophie Silver, Kornel, Asger Hautop Drewsen, zopsicle, vrtgs, Joseph Perez and Js2xxx@rustbot label A-box
View all comments
Feature gate:
#![feature(thin_box)]This is a tracking issue for
ThinBoxfor making heap allocated DSTs which only occupy 1 pointer on the stack.Public API
Note: This API is expected to grow to match
Boxas necessary over time. The initial API is the minimum needed for basic usage for error handling.Steps / History
SendandSync: ImplementSendandSyncforThinBox<T>#98595Unresolved Questions
ThinBox<T>covariant inT#98585const_allocatefleshed out and tested enough to justify it being used in thinbox's impl? Does T-lang agree we'll always have some way to procedurally generate allocations in const eval? (in contrast to declaratively via constants, statics or anonymous promoteds)