Repository navigation
Tracking Issue for pointer_bytes_offsets #96283
Description
Activity
To clarify the open question's concerns a bit: if you make a ref-slice and then cast it to a raw-slice, offsetting the raw pointer will not change the slice's len, but you can't actually turn the offset slice back into a ref, because (at least under the strictest rules) the original ref-slice has provenance over only its exact range, and so derived pointers cannot access outside the range. Creating the offset ref-slice is trying to assert provenance over additional bytes you don't have permission to access.
Similarly, trait-object-refs will make you sad if you do that.
This can only be soundly used (meaningfully) if you from_raw_parts the raw-DST from a raw pointer with provenance over the whole "array" you want to stride through.
So like, in general you need to write something like (pseudocode of the general pattern):
let array = [...]; // Steal metadata from an example or make the metadata from whole cloth let dst = &array[something]; let metadata = get_metadata(dst); // Now build a raw DST from a pointer to the whole array let raw_array_ptr = array.as_ptr(); let raw_dst = ptr::from_raw_parts(raw_array_ptr, metadata); // Yay this actually can be strided through the whole array now let next_dst = raw_dst.byte_add(offset_to_next_dst);
(CC @RalfJung since this is an interesting stacked borrows example, no need to reply if there's nothing more to say.)
Reacted by Ralf Jung, Harry Wu and waffle@Gankra actually, I think allowing these methods take pointers to DST unlocks a lot of abilities for custom DSTs. In particular
byte_subis required to implementArc::from_raw(as it accepts a possibly fat pointer toTthat needs to be offseted back toArcInner<T>).I want to propose to stabilize these methods! (can someone open an fcp?)
Stabilization report
Implementation History
- Initially these methods were implemented in Add convenience byte offset/check align functions to pointers #95643
- The only unresolved question raised since then is whatever to allow calling these methods on pointers to unsized types. I propose to resolve this to "yes, we should actually", this will unlock a lot of options for working with unsized types (see expitience report for an example).
API Summary
impl *const T { pub const unsafe fn byte_offset(self, count: isize) -> Self; pub const unsafe fn byte_add(self, count: usize) -> Self; pub const unsafe fn byte_sub(self, count: usize) -> Self; pub const fn wrapping_byte_offset(self, count: isize) -> Self; pub const fn wrapping_byte_add(self, count: usize) -> Self; pub const fn wrapping_byte_sub(self, count: usize) -> Self; pub const unsafe fn byte_offset_from<U: ?Sized>(self, origin: *const U) -> isize; } // ... and the same for` *mut T`
Experience Report
#99113, #100819 and #100030 refactored some std code into using these methods simplifying the code.
These methods allow custom
Arcimplementations to work with unsized types just as well asstd::arc::Arc, namely they allow to offset a pointer by<size of arc metadata>while keeping pointer metadata, which is required for example forArc::from_raw.This also allows to polyfil
with_metadata_ofby usingwrapping_byte_offset, which is again very useful when working with DSTs.Pondering: does
byte_offset_fromreally need its argument to be a*const T? Arguably it doesn't care what type you pass it, so it could be*const Uinstead (or*const impl Sized, I suppose).That would allow things like
ptr::addr_of!(x.b).byte_offset_from(ptr::addr_of!(x))to compile without needing an arbitrary choice of cast.EDIT: this was updated in #103489
I have a question regarding the
offset_fromfamily of functions, stabilized or not. In all those cases where it is safe to use, why not useasinteger casts, ignoring the fact that conversion from bytes to units ofTmight be required?I just had the situation today, where I needed the offset between 2 bytes slices (one is a subslice of the other) and failed to see why I should use the unsafe
offset_from. In either case, I have to be able to reason about why my code is correct, but not having to reason about why my code doesn't invoke UB seemed like a +1 for integer casts. Obviously this is just one example, and certainly not "all those cases".But I'm wondering, what are safe examples in which I should use
offset_fromas opposed to integer casts?offset_from(and particularlysub_ptr) allows LLVM to optimize more based on the unsafety. If your code is not a place where you get a material advantage from those extra optimizations, then by all means use the safe phrasings. (sub_ptris very important forslice::Iter::len, but there's very little code that's anywhere close to as perf-critical as that.)Another advantage of offset_from is that it can be used in
const fn.For some time we had safe variants of
offset_from, maybe it is worth thinking about bringing them back.- added a commit that references this issue
on Feb 10, 2023 - added a commit that references this issue
on May 2, 2023 70 remaining items
- added a commit that references this issue
on Mar 4, 2024 - added a commit that references this issue
on Apr 8, 2024 - added 3 commits that reference this issue
on Apr 27, 2024 - added a commit that references this issue
on May 10, 2024 - added a commit that references this issue
on May 22, 2024 - added 2 commits that reference this issue
on May 27, 2024 - added a commit that references this issue
on Jun 1, 2024 - added 3 commits that reference this issue
on Sep 2, 2026
Feature gates:
#![feature(pointer_byte_offsets)]#![feature(const_pointer_byte_offsets)]This is a tracking issue for the pointer_byte_offsets raw pointer conveniences like
ptr.byte_add(offset)Public API
Steps / History
byte_offset_frommore generic: Makepointer::byte_offset_frommore generic #103489[const_]pointer_byte_offsets#116205Unresolved Questions