AFIO  v2.00 late alpha
afio_v2_xxx::mapped_file_handle Class Referenceabstract

A memory mapped regular file or device. More...

#include "mapped_file_handle.hpp"

Inheritance diagram for afio_v2_xxx::mapped_file_handle:
afio_v2_xxx::file_handle afio_v2_xxx::io_handle afio_v2_xxx::fs_handle afio_v2_xxx::handle

Public Types

using dev_t = file_handle::dev_t
 
using ino_t = file_handle::ino_t
 
using path_view_type = file_handle::path_view_type
 
using path_type = io_handle::path_type
 
using extent_type = io_handle::extent_type
 
using size_type = io_handle::size_type
 
using mode = io_handle::mode
 
using creation = io_handle::creation
 
using caching = io_handle::caching
 
using flag = io_handle::flag
 
using buffer_type = io_handle::buffer_type
 
using const_buffer_type = io_handle::const_buffer_type
 
using buffers_type = io_handle::buffers_type
 
using const_buffers_type = io_handle::const_buffers_type
 
template<class T >
using io_request = io_handle::io_request< T >
 
template<class T >
using io_result = io_handle::io_result< T >
 
enum  bitfield__flag : unsigned {
  none = 0, none = 2, none = 1, unlink_on_first_close = 1U << 0U,
  disable_safety_fsyncs = 1U << 2U, disable_safety_unlinks = 1U << 3U, disable_prefetching = 1U << 4U, maximum_prefetching = 1U << 5U,
  win_disable_unlink_emulation = 1U << 24U, win_disable_sparse_file_creation = 1U << 25U, overlapped = 1U << 28U, byte_lock_insanity = 1U << 29U,
  anonymous_inode = 1U << 30U
}
 Bitwise flags which can be specified. More...
 
using unique_id_type = QUICKCPPLIB_NAMESPACE::integers128::uint128
 The unique identifier type used by this handle.
 

Public Member Functions

constexpr mapped_file_handle ()
 Default constructor.
 
 mapped_file_handle (mapped_file_handle &&o) noexcept
 Implicit move construction of mapped_file_handle permitted.
 
 mapped_file_handle (const mapped_file_handle &)=delete
 No copy construction (use clone())
 
constexpr mapped_file_handle (file_handle &&o) noexcept
 Explicit conversion from file_handle permitted.
 
 mapped_file_handle (file_handle &&o, size_type reservation) noexcept
 Explicit conversion from file_handle permitted, this overload also attempts to map the file.
 
mapped_file_handleoperator= (mapped_file_handle &&o) noexcept
 Move assignment of mapped_file_handle permitted.
 
mapped_file_handleoperator= (const mapped_file_handle &)=delete
 No copy assignment.
 
void swap (mapped_file_handle &o) noexcept
 Swap with another instance.
 
const section_handlesection () const noexcept
 The memory section this handle is using.
 
section_handlesection () noexcept
 The memory section this handle is using.
 
const map_handlemap () const noexcept
 The map this handle is using.
 
map_handlemap () noexcept
 The map this handle is using.
 
byte * address () const noexcept
 The address in memory where this mapped file resides.
 
result< extent_type > underlying_file_maximum_extent () const noexcept
 The maximum extent of the underlying file.
 
size_type capacity () const noexcept
 The address space (to be) reserved for future expansion of this file.
 
result< size_type > reserve (size_type reservation=0) noexcept
 Reserve a new amount of address space for mapping future expansion of this file. More...
 
virtual result< void > close () noexcept override
 Immediately close the native handle type managed by this handle.
 
virtual native_handle_type release () noexcept override
 Release the native handle type managed by this handle.
 
virtual io_result< const_buffers_type > barrier (io_request< const_buffers_type > reqs=io_request< const_buffers_type >(), bool wait_for_device=false, bool and_metadata=false, deadline d=deadline()) noexcept override
 
virtual result< file_handleclone (mode mode_=mode::unchanged, caching caching_=caching::unchanged, deadline d=std::chrono::seconds(30)) const noexcept override
 
result< mapped_file_handleclone (size_type reservation, mode mode_=mode::unchanged, caching caching_=caching::unchanged, deadline d=std::chrono::seconds(30)) const noexcept
 
virtual result< extent_type > maximum_extent () const noexcept override
 Return the current maximum permitted extent of the file.
 
virtual result< extent_type > truncate (extent_type newsize) noexcept override
 Resize the current maximum permitted extent of the mapped file to the given extent, avoiding any new allocation of physical storage where supported, and mapping or unmapping any new pages up to the reservation to reflect the new maximum extent. If the new size exceeds the reservation, reserve() will be called to increase the reservation. More...
 
result< extent_type > update_map () noexcept
 Efficiently update the mapping to match that of the underlying file, returning the size of the underlying file. More...
 
virtual result< extent_type > zero (extent_type offset, extent_type bytes, deadline=deadline()) noexcept override
 Efficiently zero, and possibly deallocate, data on storage. More...
 
virtual io_result< buffers_type > read (io_request< buffers_type > reqs, deadline d=deadline()) noexcept override
 Read data from the mapped file. More...
 
virtual io_result< const_buffers_type > write (io_request< const_buffers_type > reqs, deadline d=deadline()) noexcept override
 Write data to the mapped file. More...
 
void swap (file_handle &o) noexcept
 Swap with another instance.
 
void swap (handle &o) noexcept
 Swap with another instance.
 
virtual io_result< const_buffers_type > barrier (io_request< const_buffers_type > reqs=io_request< const_buffers_type >(), bool wait_for_device=false, bool and_metadata=false, deadline d=deadline()) noexcept=0
 Issue a write reordering barrier such that writes preceding the barrier will reach storage before writes after this barrier. More...
 
result< handleclone () const noexcept
 
io_serviceservice () const noexcept
 The i/o service this handle is attached to, if any.
 
virtual result< std::vector< std::pair< extent_type, extent_type > > > extents () const noexcept
 Returns a list of currently valid extents for this open file. WARNING: racy! More...
 
virtual size_t max_buffers () const noexcept
 The maximum number of buffers which a single read or write syscall can process at a time for this specific open handle. On POSIX, this is known as IOV_MAX. More...
 
virtual io_result< buffers_type > read (io_request< buffers_type > reqs, deadline d=deadline()) noexcept
 Read data from the open handle. More...
 
io_result< buffers_type > read (extent_type offset, std::initializer_list< buffer_type > lst, deadline d=deadline()) noexcept
 
virtual io_result< const_buffers_type > write (io_request< const_buffers_type > reqs, deadline d=deadline()) noexcept
 Write data to the open handle. More...
 
io_result< const_buffers_type > write (extent_type offset, std::initializer_list< const_buffer_type > lst, deadline d=deadline()) noexcept
 
virtual result< extent_guardlock (extent_type offset, extent_type bytes, bool exclusive=true, deadline d=deadline()) noexcept
 Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes through the same semantics as the underlying OS call, including any POSIX insanity present on your platform: More...
 
result< extent_guardlock (io_request< buffers_type > reqs, deadline d=deadline()) noexcept
 
result< extent_guardlock (io_request< const_buffers_type > reqs, deadline d=deadline()) noexcept
 
result< extent_guardtry_lock (extent_type offset, extent_type bytes, bool exclusive=true) noexcept
 
virtual void unlock (extent_type offset, extent_type bytes) noexcept
 Unlocks a byte range previously locked. More...
 
virtual result< path_type > current_path () const noexcept
 
bool is_valid () const noexcept
 True if the handle is valid (and usually open)
 
bool is_readable () const noexcept
 True if the handle is readable.
 
bool is_writable () const noexcept
 True if the handle is writable.
 
bool is_append_only () const noexcept
 True if the handle is append only.
 
virtual result< void > set_append_only (bool enable) noexcept
 
bool is_overlapped () const noexcept
 True if overlapped.
 
bool is_seekable () const noexcept
 True if seekable.
 
bool requires_aligned_io () const noexcept
 True if requires aligned i/o.
 
bool is_regular () const noexcept
 True if a regular file or device.
 
bool is_directory () const noexcept
 True if a directory.
 
bool is_symlink () const noexcept
 True if a symlink.
 
bool is_multiplexer () const noexcept
 True if a multiplexer like BSD kqueues, Linux epoll or Windows IOCP.
 
bool is_process () const noexcept
 True if a process.
 
bool is_section () const noexcept
 True if a memory section.
 
caching kernel_caching () const noexcept
 Kernel cache strategy used by this handle.
 
bool are_reads_from_cache () const noexcept
 True if the handle uses the kernel page cache for reads.
 
bool are_writes_durable () const noexcept
 True if writes are safely on storage on completion.
 
bool are_safety_fsyncs_issued () const noexcept
 True if issuing safety fsyncs is on.
 
flag flags () const noexcept
 The flags this handle was opened with.
 
native_handle_type native_handle () const noexcept
 The native handle used by this handle.
 
dev_t st_dev () const noexcept
 Unless flag::disable_safety_unlinks is set, the device id of the file when opened.
 
ino_t st_ino () const noexcept
 Unless flag::disable_safety_unlinks is set, the inode of the file when opened. When combined with st_dev(), forms a unique identifer on this system.
 
unique_id_type unique_id () const noexcept
 A unique identifier for this handle across the entire system. Can be used in hash tables etc.
 
virtual result< path_handleparent_path_handle (deadline d=std::chrono::seconds(30)) const noexcept
 
virtual result< void > relink (const path_handle &base, path_view_type path, bool atomic_replace=true, deadline d=std::chrono::seconds(30)) noexcept
 
virtual result< void > unlink (deadline d=std::chrono::seconds(30)) noexcept
 

Static Public Member Functions

static result< mapped_file_handlemapped_file (size_type reservation, const path_handle &base, path_view_type _path, mode _mode=mode::read, creation _creation=creation::open_existing, caching _caching=caching::all, flag flags=flag::none) noexcept
 
static result< mapped_file_handlemapped_file (const path_handle &base, path_view_type _path, mode _mode=mode::read, creation _creation=creation::open_existing, caching _caching=caching::all, flag flags=flag::none) noexcept
 
static result< mapped_file_handlemapped_random_file (size_type reservation, const path_handle &dirpath, mode _mode=mode::write, caching _caching=caching::temporary, flag flags=flag::none) noexcept
 
static result< mapped_file_handlemapped_temp_file (size_type reservation, path_view_type name=path_view_type(), mode _mode=mode::write, creation _creation=creation::if_needed, caching _caching=caching::temporary, flag flags=flag::unlink_on_first_close) noexcept
 
static result< mapped_file_handlemapped_temp_inode (const path_handle &dir=path_discovery::storage_backed_temporary_files_directory(), mode _mode=mode::write, flag flags=flag::none) noexcept
 
static result< file_handlefile (const path_handle &base, path_view_type path, mode _mode=mode::read, creation _creation=creation::open_existing, caching _caching=caching::all, flag flags=flag::none) noexcept
 
static result< file_handlerandom_file (const path_handle &dirpath, mode _mode=mode::write, caching _caching=caching::temporary, flag flags=flag::none) noexcept
 
static result< file_handletemp_file (path_view_type name=path_view_type(), mode _mode=mode::write, creation _creation=creation::if_needed, caching _caching=caching::temporary, flag flags=flag::unlink_on_first_close) noexcept
 
static result< file_handletemp_inode (const path_handle &dirh=path_discovery::storage_backed_temporary_files_directory(), mode _mode=mode::write, flag flags=flag::none) noexcept
 

Protected Member Functions

result< void > _fetch_inode () const noexcept
 Fill in _devid and _inode from the handle via fstat()
 

Protected Attributes

size_type _reservation {0}
 
section_handle _sh
 
map_handle _mh
 
io_service_service {nullptr}
 
caching _caching {caching::none}
 
flag _flags {flag::none}
 
native_handle_type _v
 
dev_t _devid {0}
 
ino_t _inode {0}
 

Detailed Description

A memory mapped regular file or device.

Cost of openingCost of i/oConcurrency and AtomicityOther remarks
file_handleLeastSyscallPOSIX guarantees (usually)Least gotcha
async_file_handleMoreMost (syscall + malloc/free + reactor)POSIX guarantees (usually)Makes no sense to use with cached i/o as it's a very expensive way to call memcpy()
mapped_file_handleMostLeastNoneCannot be used with uncached i/o

All the major OSs on all the major 64 bit CPU architectures now offer at least 127 Tb of address spaces to user mode processes. This makes feasible mapping multi-Tb files directly into memory, and thus avoiding the syscall overhead involved when reading and writing. This becames especially important with next-gen storage devices capable of Direct Access Storage (DAX) like Optane from 2018 onwards, performance via syscalls will always be but a fraction of speaking directly to the storage device via directly mapped memory.

As an example of the gains, on Microsoft Windows to read or write 1Kb using the standard syscalls takes about fifteen times longer than the exact same i/o via mapped memory. On Linux, OS X or FreeBSD the gain is considerably lower, a 1Kb i/o might only be 50% slower via syscalls than memory maps. However for lots of say 64 byte i/o, the gain of memory maps over syscalls is unsurpassable.

This class combines a file_handle with a section_handle and a map_handle to implement a fully memory mapped file_handle. The whole file is always mapped entirely into memory, and read() and write() i/o is performed directly with the map. Reads always return the original mapped data, and do not fill any buffers passed in. For obvious reasons the utility of this class on 32-bit systems is limited, but can be useful when used with smaller files.

Note that zero length files cannot be memory mapped, and writes past the maximum extent do NOT auto-extend the size of the file, rather the data written beyond the maximum valid extent has undefined kernel-specific behaviour, which includes segfaulting. You must therefore always truncate(newsize) to resize the file and its maps before you can read or write to it, and be VERY careful to not read or write beyond the maximum extent of the file.

Therefore, when a file is created or is otherwise of zero length, address() will return a null pointer. Similarly, calling truncate(0) will close the map and section handles, they will be recreated on next truncation to a non-zero size.

For better performance when handling files which are growing, there is a concept of "address space reservation" via reserve() and capacity(), which on some kernels is automatically and efficiently expanded into when the underlying file grows. The implementation asks the kernel to set out a contiguous region of pages matching that reservation, and to map the file into the beginning of the reservation. The remainder of the pages may be inaccessible and may generate a segfault, or they may automatically reflect any growth in the underlying file. This is why read() and write() only know about the reservation size, and will read and write memory up to that reservation size, without checking if the memory involved exists or not yet. You are guaranteed that address() will not return a new value unless you truncate from a bigger length to a smaller length, or you call reserve() with a new reservation or truncate() with a value bigger than the reservation.

maximum_extent() reports the last truncated length of the mapped file (possibly by any process in the system) up to the reservation limit, NOT the maximum extent of the underlying file. When you know that another process has extended the file and you wish to map the newly appended data, you can call update_map() which guarantees that the mapping your process sees is up to date, rather than relying on any kernel-specific automatic mapping. Whether automatic or enforced by update_map(), the reservation limit will not be exceeded nor will address() suddenly return something different.

It is thus up to you to detect that the reservation has been exhausted, and to reserve a new reservation which will change the value returned by address(). This entirely manual system is a bit tedious and cumbersome to use, but as mapping files is an expensive operation given TLB shootdown, we leave it up to the end user to decide when to expend the cost of mapping.

Warning
You must be cautious when the file is being extended by third parties which are not using this mapped_file_handle to write the new data. With unified page cache kernels, mixing mapped and normal i/o is generally safe except at the end of a file where race conditions and outright kernel bugs tend to abound. To avoid these, solely and exclusively use a dedicated handle configured to atomic append only to do the appends.

Member Enumeration Documentation

◆ bitfield__flag

enum afio_v2_xxx::handle::bitfield__flag : unsigned
inherited

Bitwise flags which can be specified.

Enumerator
none 

No flags.

none 

No ability to read or write anything, but can synchronise (SYNCHRONIZE or 0)

none 

No caching whatsoever, all reads and writes come from storage (i.e. O_DIRECT|O_SYNC). Align all i/o to 4Kb boundaries for this to work. flag_disable_safety_fsyncs can be used here.

unlink_on_first_close 

Unlinks the file on handle close. On POSIX, this simply unlinks whatever is pointed to by path() upon the call of close() if and only if the inode matches. On Windows, if you are on Windows 10 1709 or later, exactly the same thing occurs. If on previous editions of Windows, the file entry does not disappears but becomes unavailable for anyone else to open with an errc::resource_unavailable_try_again error return. Because this is confusing, unless the win_disable_unlink_emulation flag is also specified, this POSIX behaviour is somewhat emulated by AFIO on older Windows by renaming the file to a random name on close() causing it to appear to have been unlinked immediately.

disable_safety_fsyncs 

Some kernel caching modes have unhelpfully inconsistent behaviours in getting your data onto storage, so by default unless this flag is specified AFIO adds extra fsyncs to the following operations for the caching modes specified below: truncation of file length either explicitly or during file open. closing of the handle either explicitly or in the destructor.

Additionally on Linux only to prevent loss of file metadata: On the parent directory whenever a file might have been created. On the parent directory on file close.

This only occurs for these kernel caching modes: caching::none caching::reads caching::reads_and_metadata caching::safety_fsyncs

disable_safety_unlinks 

file_handle::unlink() could accidentally delete the wrong file if someone has renamed the open file handle since the time it was opened. To prevent this occuring, where the OS doesn't provide race free unlink-by-open-handle we compare the inode of the path we are about to unlink with that of the open handle before unlinking.

Warning
This does not prevent races where in between the time of checking the inode and executing the unlink a third party changes the item about to be unlinked. Only operating systems with a true race-free unlink syscall are race free.
disable_prefetching 

Ask the OS to disable prefetching of data. This can improve random i/o performance.

maximum_prefetching 

Ask the OS to maximise prefetching of data, possibly prefetching the entire file into kernel cache. This can improve sequential i/o performance.

win_disable_unlink_emulation 

See the documentation for unlink_on_first_close

win_disable_sparse_file_creation 

Microsoft Windows NTFS, having been created in the late 1980s, did not originally implement extents-based storage and thus could only represent sparse files via efficient compression of intermediate zeros. With NTFS v3.0 (Microsoft Windows 2000), a proper extents-based on-storage representation was added, thus allowing only 64Kb extent chunks written to be stored irrespective of whatever the maximum file extent was set to.

For various historical reasons, extents-based storage is disabled by default in newly created files on NTFS, unlike in almost every other major filing system. You have to explicitly "opt in" to extents-based storage.

As extents-based storage is nearly cost free on NTFS, AFIO by default opts in to extents-based storage for any empty file it creates. If you don't want this, you can specify this flag to prevent that happening.

overlapped 

On Windows, create any new handles with OVERLAPPED semantics.

byte_lock_insanity 

Using insane POSIX byte range locks.

anonymous_inode 

This is an inode created with no representation on the filing system.

97  {
98  none = 0, //!< No flags
99  /*! Unlinks the file on handle close. On POSIX, this simply unlinks whatever is pointed
100  to by `path()` upon the call of `close()` if and only if the inode matches. On Windows,
101  if you are on Windows 10 1709 or later, exactly the same thing occurs. If on previous
102  editions of Windows, the file entry does not disappears but becomes unavailable for
103  anyone else to open with an `errc::resource_unavailable_try_again` error return. Because this is confusing, unless the
104  `win_disable_unlink_emulation` flag is also specified, this POSIX behaviour is
105  somewhat emulated by AFIO on older Windows by renaming the file to a random name on `close()`
106  causing it to appear to have been unlinked immediately.
107  */
108  unlink_on_first_close = 1U << 0U,
109 
110  /*! Some kernel caching modes have unhelpfully inconsistent behaviours
111  in getting your data onto storage, so by default unless this flag is
112  specified AFIO adds extra fsyncs to the following operations for the
113  caching modes specified below:
114  * truncation of file length either explicitly or during file open.
115  * closing of the handle either explicitly or in the destructor.
116 
117  Additionally on Linux only to prevent loss of file metadata:
118  * On the parent directory whenever a file might have been created.
119  * On the parent directory on file close.
120 
121  This only occurs for these kernel caching modes:
122  * caching::none
123  * caching::reads
124  * caching::reads_and_metadata
125  * caching::safety_fsyncs
126  */
127  disable_safety_fsyncs = 1U << 2U,
128  /*! `file_handle::unlink()` could accidentally delete the wrong file if someone has
129  renamed the open file handle since the time it was opened. To prevent this occuring,
130  where the OS doesn't provide race free unlink-by-open-handle we compare the inode of
131  the path we are about to unlink with that of the open handle before unlinking.
132  \warning This does not prevent races where in between the time of checking the inode
133  and executing the unlink a third party changes the item about to be unlinked. Only
134  operating systems with a true race-free unlink syscall are race free.
135  */
136  disable_safety_unlinks = 1U << 3U,
137  /*! Ask the OS to disable prefetching of data. This can improve random
138  i/o performance.
139  */
140  disable_prefetching = 1U << 4U,
141  /*! Ask the OS to maximise prefetching of data, possibly prefetching the entire file
142  into kernel cache. This can improve sequential i/o performance.
143  */
144  maximum_prefetching = 1U << 5U,
145 
146  win_disable_unlink_emulation = 1U << 24U, //!< See the documentation for `unlink_on_first_close`
147  /*! Microsoft Windows NTFS, having been created in the late 1980s, did not originally
148  implement extents-based storage and thus could only represent sparse files via
149  efficient compression of intermediate zeros. With NTFS v3.0 (Microsoft Windows 2000),
150  a proper extents-based on-storage representation was added, thus allowing only 64Kb
151  extent chunks written to be stored irrespective of whatever the maximum file extent
152  was set to.
153 
154  For various historical reasons, extents-based storage is disabled by default in newly
155  created files on NTFS, unlike in almost every other major filing system. You have to
156  explicitly "opt in" to extents-based storage.
157 
158  As extents-based storage is nearly cost free on NTFS, AFIO by default opts in to
159  extents-based storage for any empty file it creates. If you don't want this, you
160  can specify this flag to prevent that happening.
161  */
163 
164  // NOTE: IF UPDATING THIS UPDATE THE std::ostream PRINTER BELOW!!!
165 
166  overlapped = 1U << 28U, //!< On Windows, create any new handles with OVERLAPPED semantics
167  byte_lock_insanity = 1U << 29U, //!< Using insane POSIX byte range locks
168  anonymous_inode = 1U << 30U //!< This is an inode created with no representation on the filing system
169  }
See the documentation for unlink_on_first_close
Definition: handle.hpp:146
This is an inode created with no representation on the filing system.
Definition: handle.hpp:168
On Windows, create any new handles with OVERLAPPED semantics.
Definition: handle.hpp:166
No flags.
Definition: handle.hpp:98
Using insane POSIX byte range locks.
Definition: handle.hpp:167

Member Function Documentation

◆ barrier()

virtual io_result<const_buffers_type> afio_v2_xxx::io_handle::barrier ( io_request< const_buffers_type reqs = io_requestconst_buffers_type >(),
bool  wait_for_device = false,
bool  and_metadata = false,
deadline  d = deadline() 
)
pure virtualnoexceptinherited

Issue a write reordering barrier such that writes preceding the barrier will reach storage before writes after this barrier.

Warning
Assume that this call is a no-op. It is not reliably implemented in many common use cases, for example if your code is running inside a LXC container, or if the user has mounted the filing system with non-default options. Instead open the handle with caching::reads which means that all writes form a strict sequential order not completing until acknowledged by the storage device. Filing system can and do use different algorithms to give much better performance with caching::reads, some (e.g. ZFS) spectacularly better.
Let me repeat again: consider this call to be a hint to poke the kernel with a stick to go start to do some work sooner rather than later. It may be ignored entirely.
For portability, you can only assume that barriers write order for a single handle instance. You cannot assume that barriers write order across multiple handles to the same inode, or across processes.
Returns
The buffers barriered, which may not be the buffers input. The size of each scatter-gather buffer is updated with the number of bytes of that buffer barriered.
Parameters
reqsA scatter-gather and offset request for what range to barrier. May be ignored on some platforms which always write barrier the entire file. Supplying a default initialised reqs write barriers the entire file.
wait_for_deviceTrue if you want the call to wait until data reaches storage and that storage has acknowledged the data is physically written. Slow.
and_metadataTrue if you want the call to sync the metadata for retrieving the writes before the barrier after a sudden power loss event. Slow. Setting this to false enables much faster performance, especially on non-volatile memory.
dAn optional deadline by which the i/o must complete, else it is cancelled. Note function may return significantly after this deadline if the i/o takes long to cancel.
Errors returnable
Any of the values POSIX fdatasync() or Windows NtFlushBuffersFileEx() can return.
Memory Allocations
None.

◆ clone() [1/2]

result<handle> afio_v2_xxx::handle::clone ( ) const
inlinenoexceptinherited

Clone this handle (copy constructor is disabled to avoid accidental copying)

Errors returnable
Any of the values POSIX dup() or DuplicateHandle() can return.

◆ clone() [2/2]

virtual result<file_handle> afio_v2_xxx::mapped_file_handle::clone ( mode  mode_ = mode::unchanged,
caching  caching_ = caching::unchanged,
deadline  d = std::chrono::seconds(30) 
) const
inlineoverridevirtualnoexcept

Clone this handle (copy constructor is disabled to avoid accidental copying), optionally race free reopening the handle with different access or caching.

Microsoft Windows provides a syscall for cloning an existing handle but with new access. On POSIX, if not changing the mode, we change caching via fcntl(), if changing the mode we must loop calling current_path(), trying to open the path returned and making sure it is the same inode.

Errors returnable
Any of the values POSIX dup() or DuplicateHandle() can return.
Memory Allocations
On POSIX if changing the mode, we must loop calling current_path() and trying to open the path returned. Thus many allocations may occur.

Reimplemented from afio_v2_xxx::file_handle.

325  {
326  OUTCOME_TRY(fh, file_handle::clone(mode_, caching_, d));
327  mapped_file_handle ret(std::move(fh), _reservation);
328  return static_cast<file_handle &&>(ret);
329  }
constexpr mapped_file_handle()
Default constructor.
Definition: mapped_file_handle.hpp:134
result< handle > clone() const noexcept
constexpr file_handle()
Default constructor.
Definition: file_handle.hpp:82

◆ current_path()

virtual result<path_type> afio_v2_xxx::handle::current_path ( ) const
inlinevirtualnoexceptinherited

Returns the current path of the open handle as said by the operating system. Note that you are NOT guaranteed that any path refreshed bears any resemblance to the original, some operating systems will return some different path which still reaches the same inode via some other route e.g. hardlinks, dereferenced symbolic links, etc. Windows and Linux correctly track changes to the specific path the handle was opened with, not getting confused by other hard links. MacOS nearly gets it right, but under some circumstances e.g. renaming may switch to a different hard link's path which is almost certainly a bug.

If AFIO was not able to determine the current path for this open handle e.g. the inode has been unlinked, it returns an empty path. Be aware that FreeBSD can return an empty (deleted) path for file inodes no longer cached by the kernel path cache, AFIO cannot detect the difference. FreeBSD will also return any path leading to the inode if it is hard linked. FreeBSD does implement path retrieval for directory inodes correctly however, and see algorithm::stablized_path<T> for a handle adapter which makes use of that.

On Linux if /proc is not mounted, this call fails with an error. All APIs in AFIO which require the use of current_path() can be told to not use it e.g. flag::disable_safety_unlinks. It is up to you to detect if current_path() is not working, and to change how you call AFIO appropriately.

Warning
This call is expensive, it always asks the kernel for the current path, and no checking is done to ensure what the kernel returns is accurate or even sensible. Be aware that despite these precautions, paths are unstable and can change randomly at any moment. Most code written to use absolute file systems paths is racy, so don't do it, use path_handle to fix a base location on the file system and work from that anchor instead!
Memory Allocations
At least one malloc for the path_type, likely several more.
See also
algorithm::cached_parent_handle_adapter<T> which overrides this with an implementation based on retrieving the current path of a cached handle to the parent directory. On platforms with instability or failure to retrieve the correct current path for regular files, the cached parent handle adapter works around the problem by taking advantage of directory inodes not having the same instability problems on any platform.

◆ extents()

virtual result<std::vector<std::pair<extent_type, extent_type> > > afio_v2_xxx::file_handle::extents ( ) const
inlinevirtualnoexceptinherited

Returns a list of currently valid extents for this open file. WARNING: racy!

Returns
A vector of pairs of extent offset + extent length representing the valid extents in this file. Filing systems which do not support extents return a single extent matching the length of the file rather than returning an error.

◆ file()

static result<file_handle> afio_v2_xxx::file_handle::file ( const path_handle base,
path_view_type  path,
mode  _mode = mode::read,
creation  _creation = creation::open_existing,
caching  _caching = caching::all,
flag  flags = flag::none 
)
inlinestaticnoexceptinherited

Create a file handle opening access to a file on path

Parameters
baseHandle to a base location on the filing system. Pass {} to indicate that path will be absolute.
pathThe path relative to base to open.
_modeHow to open the file.
_creationHow to create the file.
_cachingHow to ask the kernel to cache the file.
flagsAny additional custom behaviours.
Errors returnable
Any of the values POSIX open() or CreateFile() can return.

◆ lock() [1/3]

virtual result<extent_guard> afio_v2_xxx::io_handle::lock ( extent_type  offset,
extent_type  bytes,
bool  exclusive = true,
deadline  d = deadline() 
)
inlinevirtualnoexceptinherited

Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes through the same semantics as the underlying OS call, including any POSIX insanity present on your platform:

  • Any fd closed on an inode must release all byte range locks on that inode for all other fds. If your OS isn't new enough to support the non-insane lock API, flag::byte_lock_insanity will be set in flags() after the first call to this function.
  • Threads replace each other's locks, indeed locks replace each other's locks.

You almost cetainly should use your choice of an algorithm::shared_fs_mutex::* instead of this as those are more portable and performant.

Warning
This is a low-level API which you should not use directly in portable code. Another issue is that atomic lock upgrade/downgrade, if your platform implements that (you should assume it does not in portable code), means that on POSIX you need to release the old extent_guard after creating a new one over the same byte range, otherwise the old extent_guard's destructor will simply unlock the range entirely. On Windows however upgrade/downgrade locks overlay, so on that platform you must not release the old extent_guard. Look into algorithm::shared_fs_mutex::safe_byte_ranges for a portable solution.
Returns
An extent guard, the destruction of which will call unlock().
Parameters
offsetThe offset to lock. Note that on POSIX the top bit is always cleared before use as POSIX uses signed transport for offsets. If you want an advisory rather than mandatory lock on Windows, one technique is to force top bit set so the region you lock is not the one you will i/o - obviously this reduces maximum file size to (2^63)-1.
bytesThe number of bytes to lock. Zero means lock the entire file using any more efficient alternative algorithm where available on your platform (specifically, on BSD and OS X use flock() for non-insane semantics).
exclusiveWhether the lock is to be exclusive.
dAn optional deadline by which the lock must complete, else it is cancelled.
Errors returnable
Any of the values POSIX fcntl() can return, errc::timed_out, errc::not_supported may be returned if deadline i/o is not possible with this particular handle configuration (e.g. non-overlapped HANDLE on Windows).
Memory Allocations
The default synchronous implementation in file_handle performs no memory allocation. The asynchronous implementation in async_file_handle performs one calloc and one free.

◆ lock() [2/3]

result<extent_guard> afio_v2_xxx::io_handle::lock ( io_request< buffers_type reqs,
deadline  d = deadline() 
)
inlinenoexceptinherited

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

424  {
425  size_t bytes = 0;
426  for(auto &i : reqs.buffers)
427  {
428  if(bytes + i.len < bytes)
429  {
430  return errc::value_too_large;
431  }
432  bytes += i.len;
433  }
434  return lock(reqs.offset, bytes, false, d);
435  }
virtual result< extent_guard > lock(extent_type offset, extent_type bytes, bool exclusive=true, deadline d=deadline()) noexcept
Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes throu...

◆ lock() [3/3]

result<extent_guard> afio_v2_xxx::io_handle::lock ( io_request< const_buffers_type reqs,
deadline  d = deadline() 
)
inlinenoexceptinherited

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

438  {
439  size_t bytes = 0;
440  for(auto &i : reqs.buffers)
441  {
442  if(bytes + i.len < bytes)
443  {
444  return errc::value_too_large;
445  }
446  bytes += i.len;
447  }
448  return lock(reqs.offset, bytes, true, d);
449  }
virtual result< extent_guard > lock(extent_type offset, extent_type bytes, bool exclusive=true, deadline d=deadline()) noexcept
Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes throu...

◆ mapped_file() [1/2]

static result<mapped_file_handle> afio_v2_xxx::mapped_file_handle::mapped_file ( size_type  reservation,
const path_handle base,
path_view_type  _path,
mode  _mode = mode::read,
creation  _creation = creation::open_existing,
caching  _caching = caching::all,
flag  flags = flag::none 
)
inlinestaticnoexcept

Create a memory mapped file handle opening access to a file on path.

Parameters
reservationThe number of bytes to reserve for later expansion when mapping. Zero means reserve only the current file length.
baseHandle to a base location on the filing system. Pass {} to indicate that path will be absolute.
_pathThe path relative to base to open.
_modeHow to open the file.
_creationHow to create the file.
_cachingHow to ask the kernel to cache the file.
flagsAny additional custom behaviours.

Note that if the file is currently zero sized, no mapping occurs now, but later when truncate() or update_map() is called.

Errors returnable
Any of the values which the constructors for file_handle, section_handle and map_handle can return.
190  {
191  if(_mode == mode::append)
192  {
193  return errc::invalid_argument;
194  }
195  OUTCOME_TRY(fh, file_handle::file(base, _path, _mode, _creation, _caching, flags));
196  switch(_creation)
197  {
198  default:
199  {
200  // Attempt mapping now (may silently fail if file is empty)
201  mapped_file_handle mfh(std::move(fh), reservation);
202  return {std::move(mfh)};
203  }
204  case creation::only_if_not_exist:
205  case creation::truncate:
206  {
207  // Don't attempt mapping now as file will be empty
208  mapped_file_handle mfh(std::move(fh));
209  mfh._reservation = reservation;
210  return {std::move(mfh)};
211  }
212  }
213  }
constexpr mapped_file_handle()
Default constructor.
Definition: mapped_file_handle.hpp:134
static result< file_handle > file(const path_handle &base, path_view_type path, mode _mode=mode::read, creation _creation=creation::open_existing, caching _caching=caching::all, flag flags=flag::none) noexcept
Atomically truncate on open, leaving creation date unmodified.
flag flags() const noexcept
The flags this handle was opened with.
Definition: handle.hpp:314
All mainstream OSs and CIFS guarantee this is atomic with respect to all other appenders (FILE_APPEND...

◆ mapped_file() [2/2]

static result<mapped_file_handle> afio_v2_xxx::mapped_file_handle::mapped_file ( const path_handle base,
path_view_type  _path,
mode  _mode = mode::read,
creation  _creation = creation::open_existing,
caching  _caching = caching::all,
flag  flags = flag::none 
)
inlinestaticnoexcept

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

216 { return mapped_file(0, base, _path, _mode, _creation, _caching, flags); }
flag flags() const noexcept
The flags this handle was opened with.
Definition: handle.hpp:314
static result< mapped_file_handle > mapped_file(size_type reservation, const path_handle &base, path_view_type _path, mode _mode=mode::read, creation _creation=creation::open_existing, caching _caching=caching::all, flag flags=flag::none) noexcept
Definition: mapped_file_handle.hpp:189

◆ mapped_random_file()

static result<mapped_file_handle> afio_v2_xxx::mapped_file_handle::mapped_random_file ( size_type  reservation,
const path_handle dirpath,
mode  _mode = mode::write,
caching  _caching = caching::temporary,
flag  flags = flag::none 
)
inlinestaticnoexcept

Create an mapped file handle creating a randomly named file on a path. The file is opened exclusively with creation::only_if_not_exist so it will never collide with nor overwrite any existing file. Note also that caching defaults to temporary which hints to the OS to only flush changes to physical storage as lately as possible.

Errors returnable
Any of the values POSIX open() or CreateFile() can return.
228  {
229  try
230  {
231  for(;;)
232  {
233  auto randomname = utils::random_string(32);
234  randomname.append(".random");
235  result<mapped_file_handle> ret = mapped_file(reservation, dirpath, randomname, _mode, creation::only_if_not_exist, _caching, flags);
236  if(ret || (!ret && ret.error() != errc::file_exists))
237  {
238  return ret;
239  }
240  }
241  }
242  catch(...)
243  {
244  return error_from_exception();
245  }
246  }
std::string random_string(size_t randomlen)
Returns a cryptographically random string capable of being used as a filename. Essentially random_fil...
Definition: utils.hpp:134
flag flags() const noexcept
The flags this handle was opened with.
Definition: handle.hpp:314
static result< mapped_file_handle > mapped_file(size_type reservation, const path_handle &base, path_view_type _path, mode _mode=mode::read, creation _creation=creation::open_existing, caching _caching=caching::all, flag flags=flag::none) noexcept
Definition: mapped_file_handle.hpp:189

◆ mapped_temp_file()

static result<mapped_file_handle> afio_v2_xxx::mapped_file_handle::mapped_temp_file ( size_type  reservation,
path_view_type  name = path_view_type(),
mode  _mode = mode::write,
creation  _creation = creation::if_needed,
caching  _caching = caching::temporary,
flag  flags = flag::unlink_on_first_close 
)
inlinestaticnoexcept

Create a mapped file handle creating the named file on some path which the OS declares to be suitable for temporary files. Most OSs are very lazy about flushing changes made to these temporary files. Note the default flags are to have the newly created file deleted on first handle close. Note also that an empty name is equivalent to calling mapped_random_file(path_discovery::storage_backed_temporary_files_directory()) and the creation parameter is ignored.

Note
If the temporary file you are creating is not going to have its path sent to another process for usage, this is the WRONG function to use. Use temp_inode() instead, it is far more secure.
Errors returnable
Any of the values POSIX open() or CreateFile() can return.
264  {
266  return name.empty() ? mapped_random_file(reservation, tempdirh, _mode, _caching, flags) : mapped_file(reservation, tempdirh, name, _mode, _creation, _caching, flags);
267  }
const path_handle & storage_backed_temporary_files_directory() noexcept
Returns a reference to an open handle to a verified temporary directory where files created are store...
static result< mapped_file_handle > mapped_random_file(size_type reservation, const path_handle &dirpath, mode _mode=mode::write, caching _caching=caching::temporary, flag flags=flag::none) noexcept
Definition: mapped_file_handle.hpp:227
flag flags() const noexcept
The flags this handle was opened with.
Definition: handle.hpp:314
static result< mapped_file_handle > mapped_file(size_type reservation, const path_handle &base, path_view_type _path, mode _mode=mode::read, creation _creation=creation::open_existing, caching _caching=caching::all, flag flags=flag::none) noexcept
Definition: mapped_file_handle.hpp:189

◆ mapped_temp_inode()

static result<mapped_file_handle> afio_v2_xxx::mapped_file_handle::mapped_temp_inode ( const path_handle dir = path_discovery::storage_backed_temporary_files_directory(),
mode  _mode = mode::write,
flag  flags = flag::none 
)
inlinestaticnoexcept

Securely create a mapped file handle creating a temporary anonymous inode in the filesystem referred to by dirpath. The inode created has no name nor accessible path on the filing system and ceases to exist as soon as the last handle is closed, making it ideal for use as a temporary file where other processes do not need to have access to its contents via some path on the filing system (a classic use case is for backing shared memory maps).

Errors returnable
Any of the values POSIX open() or CreateFile() can return.
280  {
281  OUTCOME_TRY(v, file_handle::temp_inode(dir, _mode, flags));
282  mapped_file_handle ret(std::move(v));
283  return std::move(ret);
284  }
constexpr mapped_file_handle()
Default constructor.
Definition: mapped_file_handle.hpp:134
static result< file_handle > temp_inode(const path_handle &dirh=path_discovery::storage_backed_temporary_files_directory(), mode _mode=mode::write, flag flags=flag::none) noexcept
flag flags() const noexcept
The flags this handle was opened with.
Definition: handle.hpp:314

◆ max_buffers()

virtual size_t afio_v2_xxx::io_handle::max_buffers ( ) const
inlinevirtualnoexceptinherited

The maximum number of buffers which a single read or write syscall can process at a time for this specific open handle. On POSIX, this is known as IOV_MAX.

Note that the actual number of buffers accepted for a read or a write may be significantly lower than this system-defined limit, depending on available resources. The read() or write() call will return the buffers accepted.

Note also that some OSs will error out if you supply more than this limit to read() or write(), but other OSs do not. Some OSs guarantee that each i/o syscall has effects atomically visible or not to other i/o, other OSs do not.

Microsoft Windows and OS X does not implement scatter-gather file i/o syscalls. Thus this function will always return 1 in that situation.

◆ parent_path_handle()

virtual result<path_handle> afio_v2_xxx::fs_handle::parent_path_handle ( deadline  d = std::chrono::seconds(30)) const
inlinevirtualnoexceptinherited

Obtain a handle to the path currently containing this handle's file entry.

Warning
This call is racy and can result in the wrong path handle being returned. Note that unless flag::disable_safety_unlinks is set, this implementation opens a path_handle to the source containing directory, then checks if the file entry within has the same inode as the open file handle. It will retry this matching until success until the deadline given.
Memory Allocations
Calls current_path() and thus is both expensive and calls malloc many times.
See also
algorithm::cached_parent_handle_adapter<T> which overrides this with a zero cost implementation, thus making unlinking and relinking very considerably quicker.

◆ random_file()

static result<file_handle> afio_v2_xxx::file_handle::random_file ( const path_handle dirpath,
mode  _mode = mode::write,
caching  _caching = caching::temporary,
flag  flags = flag::none 
)
inlinestaticnoexceptinherited

Create a file handle creating a randomly named file on a path. The file is opened exclusively with creation::only_if_not_exist so it will never collide with nor overwrite any existing file. Note also that caching defaults to temporary which hints to the OS to only flush changes to physical storage as lately as possible.

Errors returnable
Any of the values POSIX open() or CreateFile() can return.
136  {
137  try
138  {
139  for(;;)
140  {
141  auto randomname = utils::random_string(32);
142  randomname.append(".random");
143  result<file_handle> ret = file(dirpath, randomname, _mode, creation::only_if_not_exist, _caching, flags);
144  if(ret || (!ret && ret.error() != errc::file_exists))
145  {
146  return ret;
147  }
148  }
149  }
150  catch(...)
151  {
152  return error_from_exception();
153  }
154  }
static result< file_handle > file(const path_handle &base, path_view_type path, mode _mode=mode::read, creation _creation=creation::open_existing, caching _caching=caching::all, flag flags=flag::none) noexcept
std::string random_string(size_t randomlen)
Returns a cryptographically random string capable of being used as a filename. Essentially random_fil...
Definition: utils.hpp:134
flag flags() const noexcept
The flags this handle was opened with.
Definition: handle.hpp:314

◆ read() [1/3]

virtual io_result<buffers_type> afio_v2_xxx::io_handle::read ( io_request< buffers_type reqs,
deadline  d = deadline() 
)
inlinevirtualnoexceptinherited

Read data from the open handle.

Warning
Depending on the implementation backend, very different buffers may be returned than you supplied. You should always use the buffers returned and assume that they point to different memory and that each buffer's size will have changed.
Returns
The buffers read, which may not be the buffers input. The size of each scatter-gather buffer is updated with the number of bytes of that buffer transferred, and the pointer to the data may be completely different to what was submitted (e.g. it may point into a memory map).
Parameters
reqsA scatter-gather and offset request.
dAn optional deadline by which the i/o must complete, else it is cancelled. Note function may return significantly after this deadline if the i/o takes long to cancel.
Errors returnable
Any of the values POSIX read() can return, errc::timed_out, errc::operation_canceled. errc::not_supported may be returned if deadline i/o is not possible with this particular handle configuration (e.g. reading from regular files on POSIX or reading from a non-overlapped HANDLE on Windows).
Memory Allocations
The default synchronous implementation in file_handle performs no memory allocation. The asynchronous implementation in async_file_handle performs one calloc and one free.

◆ read() [2/3]

io_result<buffers_type> afio_v2_xxx::io_handle::read ( extent_type  offset,
std::initializer_list< buffer_type lst,
deadline  d = deadline() 
)
inlinenoexceptinherited

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

236  {
237  buffer_type *_reqs = reinterpret_cast<buffer_type *>(alloca(sizeof(buffer_type) * lst.size()));
238  memcpy(_reqs, lst.begin(), sizeof(buffer_type) * lst.size());
239  io_request<buffers_type> reqs(buffers_type(_reqs, lst.size()), offset);
240  return read(reqs, d);
241  }
span< buffer_type > buffers_type
The scatter buffers type used by this handle. Guaranteed to be TrivialType apart from construction...
Definition: io_handle.hpp:86
virtual io_result< buffers_type > read(io_request< buffers_type > reqs, deadline d=deadline()) noexcept
Read data from the open handle.

◆ read() [3/3]

virtual io_result<buffers_type> afio_v2_xxx::mapped_file_handle::read ( io_request< buffers_type >  reqs,
deadline  d = deadline() 
)
inlineoverridevirtualnoexcept

Read data from the mapped file.

Note
Because this implementation never copies memory, you can pass in buffers with a null address. As this function never reads any memory, no attempt to trap signal raises can be made, this falls onto the user of this function. See QUICKCPPLIB_NAMESPACE::signal_guard for a helper function.
Returns
The buffers read, which will never be the buffers input, because they will point into the mapped view. The size of each scatter-gather buffer is updated with the number of bytes of that buffer transferred.
Parameters
reqsA scatter-gather and offset request.
dIgnored.
Errors returnable
None, though the various signals and structured exception throws common to using memory maps may occur.
Memory Allocations
None.
392 { return _mh.read(reqs, d); }
virtual io_result< buffers_type > read(io_request< buffers_type > reqs, deadline d=deadline()) noexcept override
Read data from the mapped view.

◆ relink()

virtual result<void> afio_v2_xxx::fs_handle::relink ( const path_handle base,
path_view_type  path,
bool  atomic_replace = true,
deadline  d = std::chrono::seconds(30) 
)
inlinevirtualnoexceptinherited

Relinks the current path of this open handle to the new path specified. If atomic_replace is true, the relink atomically and silently replaces any item at the new path specified. This operation is both atomic and silent matching POSIX behaviour even on Microsoft Windows where no Win32 API can match POSIX semantics.

Warning
Some operating systems provide a race free syscall for renaming an open handle (Windows). On all other operating systems this call is racy and can result in the wrong file entry being relinked. Note that unless flag::disable_safety_unlinks is set, this implementation opens a path_handle to the source containing directory first, then checks before relinking that the item about to be relinked has the same inode as the open file handle. It will retry this matching until success until the deadline given. This should prevent most unmalicious accidental loss of data.
Parameters
baseBase for any relative path.
pathThe relative or absolute new path to relink to.
atomic_replaceAtomically replace the destination if a file entry already is present there. Choosing false for this will fail if a file entry is already present at the destination, and may not be an atomic operation on some platforms (i.e. both the old and new names may be linked to the same inode for a very short period of time). Windows and recent Linuxes are always atomic.
dThe deadline by which the matching of the containing directory to the open handle's inode must succeed, else errc::timed_out will be returned.
Memory Allocations
Except on platforms with race free syscalls for renaming open handles (Windows), calls current_path() via parent_path_handle() and thus is both expensive and calls malloc many times.

◆ reserve()

result<size_type> afio_v2_xxx::mapped_file_handle::reserve ( size_type  reservation = 0)
inlinenoexcept

Reserve a new amount of address space for mapping future expansion of this file.

Parameters
reservationThe number of bytes of virtual address space to reserve. Zero means reserve the current length of the underlying file.

Note that this is an expensive call, and address() may return a different value afterwards. This call will fail if the underlying file has zero length.

◆ set_append_only()

virtual result<void> afio_v2_xxx::handle::set_append_only ( bool  enable)
inlinevirtualnoexceptinherited

Changes whether this handle is append only or not.

Warning
On Windows this is implemented as a bit of a hack to make it fast like on POSIX, so make sure you open the handle for read/write originally. Note unlike on POSIX the append_only disposition will be the only one toggled, seekable and readable will remain turned on.
Errors returnable
Whatever POSIX fcntl() returns. On Windows nothing is changed on the handle.
Memory Allocations
No memory allocation.

◆ temp_file()

static result<file_handle> afio_v2_xxx::file_handle::temp_file ( path_view_type  name = path_view_type(),
mode  _mode = mode::write,
creation  _creation = creation::if_needed,
caching  _caching = caching::temporary,
flag  flags = flag::unlink_on_first_close 
)
inlinestaticnoexceptinherited

Create a file handle creating the named file on some path which the OS declares to be suitable for temporary files. Most OSs are very lazy about flushing changes made to these temporary files. Note the default flags are to have the newly created file deleted on first handle close. Note also that an empty name is equivalent to calling random_file(path_discovery::storage_backed_temporary_files_directory()) and the creation parameter is ignored.

Note
If the temporary file you are creating is not going to have its path sent to another process for usage, this is the WRONG function to use. Use temp_inode() instead, it is far more secure.
Errors returnable
Any of the values POSIX open() or CreateFile() can return.
172  {
174  return name.empty() ? random_file(tempdirh, _mode, _caching, flags) : file(tempdirh, name, _mode, _creation, _caching, flags);
175  }
const path_handle & storage_backed_temporary_files_directory() noexcept
Returns a reference to an open handle to a verified temporary directory where files created are store...
static result< file_handle > file(const path_handle &base, path_view_type path, mode _mode=mode::read, creation _creation=creation::open_existing, caching _caching=caching::all, flag flags=flag::none) noexcept
static result< file_handle > random_file(const path_handle &dirpath, mode _mode=mode::write, caching _caching=caching::temporary, flag flags=flag::none) noexcept
Definition: file_handle.hpp:135
flag flags() const noexcept
The flags this handle was opened with.
Definition: handle.hpp:314

◆ temp_inode()

static result<file_handle> afio_v2_xxx::file_handle::temp_inode ( const path_handle dirh = path_discovery::storage_backed_temporary_files_directory(),
mode  _mode = mode::write,
flag  flags = flag::none 
)
inlinestaticnoexceptinherited

Securely create a file handle creating a temporary anonymous inode in the filesystem referred to by dirpath. The inode created has no name nor accessible path on the filing system and ceases to exist as soon as the last handle is closed, making it ideal for use as a temporary file where other processes do not need to have access to its contents via some path on the filing system (a classic use case is for backing shared memory maps).

Errors returnable
Any of the values POSIX open() or CreateFile() can return.

◆ truncate()

virtual result<extent_type> afio_v2_xxx::mapped_file_handle::truncate ( extent_type  newsize)
inlineoverridevirtualnoexcept

Resize the current maximum permitted extent of the mapped file to the given extent, avoiding any new allocation of physical storage where supported, and mapping or unmapping any new pages up to the reservation to reflect the new maximum extent. If the new size exceeds the reservation, reserve() will be called to increase the reservation.

Note that on extents based filing systems this will succeed even if there is insufficient free space on the storage medium. Only when pages are written to will the lack of sufficient free space be realised, resulting in an operating system specific exception.

Note
On Microsoft Windows you cannot shrink a file with a section handle open on it in any process in the system. We therefore always destroy the internal map and section before truncating, and then recreate the map and section afterwards if the new size is not zero. address() therefore may change. You will need to ensure all other users of the same file close their section and map handles before any process can shrink the underlying file.
Returns
The bytes actually truncated to.
Parameters
newsizeThe bytes to truncate the file to. Zero causes the maps to be closed before truncation.

Reimplemented from afio_v2_xxx::file_handle.

◆ try_lock()

result<extent_guard> afio_v2_xxx::io_handle::try_lock ( extent_type  offset,
extent_type  bytes,
bool  exclusive = true 
)
inlinenoexceptinherited

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

421 { return lock(offset, bytes, exclusive, deadline(std::chrono::seconds(0))); }
virtual result< extent_guard > lock(extent_type offset, extent_type bytes, bool exclusive=true, deadline d=deadline()) noexcept
Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes throu...

◆ unlink()

virtual result<void> afio_v2_xxx::fs_handle::unlink ( deadline  d = std::chrono::seconds(30))
inlinevirtualnoexceptinherited

Unlinks the current path of this open handle, causing its entry to immediately disappear from the filing system. On Windows before Windows 10 1709 unless flag::win_disable_unlink_emulation is set, this behaviour is simulated by renaming the file to something random and setting its delete-on-last-close flag. Note that Windows may prevent the renaming of a file in use by another process, if so it will NOT be renamed. After the next handle to that file closes, it will become permanently unopenable by anyone else until the last handle is closed, whereupon the entry will be eventually removed by the operating system.

Warning
Some operating systems provide a race free syscall for unlinking an open handle (Windows). On all other operating systems this call is racy and can result in the wrong file entry being unlinked. Note that unless flag::disable_safety_unlinks is set, this implementation opens a path_handle to the containing directory first, then checks that the item about to be unlinked has the same inode as the open file handle. It will retry this matching until success until the deadline given. This should prevent most unmalicious accidental loss of data.
Parameters
dThe deadline by which the matching of the containing directory to the open handle's inode must succeed, else errc::timed_out will be returned.
Memory Allocations
Except on platforms with race free syscalls for unlinking open handles (Windows), calls current_path() and thus is both expensive and calls malloc many times. On Windows, also calls current_path() if flag::disable_safety_unlinks is not set.

◆ unlock()

virtual void afio_v2_xxx::io_handle::unlock ( extent_type  offset,
extent_type  bytes 
)
inlinevirtualnoexceptinherited

Unlocks a byte range previously locked.

Parameters
offsetThe offset to unlock. This should be an offset previously locked.
bytesThe number of bytes to unlock. This should be a byte extent previously locked.
Errors returnable
Any of the values POSIX fcntl() can return.
Memory Allocations
None.

◆ update_map()

result<extent_type> afio_v2_xxx::mapped_file_handle::update_map ( )
inlinenoexcept

Efficiently update the mapping to match that of the underlying file, returning the size of the underlying file.

This call is often considerably less heavyweight than truncate(newsize), and should be used where possible.

If the internal section and map handle are invalid, they are restored unless the underlying file is zero length.

◆ write() [1/3]

virtual io_result<const_buffers_type> afio_v2_xxx::io_handle::write ( io_request< const_buffers_type reqs,
deadline  d = deadline() 
)
inlinevirtualnoexceptinherited

Write data to the open handle.

Warning
Depending on the implementation backend, not all of the buffers input may be written and the some buffers at the end of the returned buffers may return with zero bytes written. For example, with a zeroed deadline, some backends may only consume as many buffers as the system has available write slots for, thus for those backends this call is "non-blocking" in the sense that it will return immediately even if it could not schedule a single buffer write. Another example is that some implementations will not auto-extend the length of a file when a write exceeds the maximum extent, you will need to issue a truncate(newsize) first.
Returns
The buffers written, which may not be the buffers input. The size of each scatter-gather buffer is updated with the number of bytes of that buffer transferred.
Parameters
reqsA scatter-gather and offset request.
dAn optional deadline by which the i/o must complete, else it is cancelled. Note function may return significantly after this deadline if the i/o takes long to cancel.
Errors returnable
Any of the values POSIX write() can return, errc::timed_out, errc::operation_canceled. errc::not_supported may be returned if deadline i/o is not possible with this particular handle configuration (e.g. writing to regular files on POSIX or writing to a non-overlapped HANDLE on Windows).
Memory Allocations
The default synchronous implementation in file_handle performs no memory allocation. The asynchronous implementation in async_file_handle performs one calloc and one free.

◆ write() [2/3]

io_result<const_buffers_type> afio_v2_xxx::io_handle::write ( extent_type  offset,
std::initializer_list< const_buffer_type lst,
deadline  d = deadline() 
)
inlinenoexceptinherited

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

269  {
270  const_buffer_type *_reqs = reinterpret_cast<const_buffer_type *>(alloca(sizeof(const_buffer_type) * lst.size()));
271  memcpy(_reqs, lst.begin(), sizeof(const_buffer_type) * lst.size());
272  io_request<const_buffers_type> reqs(const_buffers_type(_reqs, lst.size()), offset);
273  return write(reqs, d);
274  }
span< const_buffer_type > const_buffers_type
The gather buffers type used by this handle. Guaranteed to be TrivialType apart from construction...
Definition: io_handle.hpp:88
virtual io_result< const_buffers_type > write(io_request< const_buffers_type > reqs, deadline d=deadline()) noexcept
Write data to the open handle.

◆ write() [3/3]

virtual io_result<const_buffers_type> afio_v2_xxx::mapped_file_handle::write ( io_request< const_buffers_type >  reqs,
deadline  d = deadline() 
)
inlineoverridevirtualnoexcept

Write data to the mapped file.

Note
This call traps signals and structured exception throws using QUICKCPPLIB_NAMESPACE::signal_guard. Instantiating a QUICKCPPLIB_NAMESPACE::signal_guard_install somewhere much higher up in the call stack will improve performance enormously. The signal guard may cost less than 100 CPU cycles depending on how you configure it. If you don't want the guard, you can write memory directly using address().
Returns
The buffers written, which will never be the buffers input because they will point at where the data was copied into the mapped view. The size of each scatter-gather buffer is updated with the number of bytes of that buffer transferred.
Parameters
reqsA scatter-gather and offset request.
dIgnored.
Errors returnable
If during the attempt to write the buffers to the map a SIGBUS or EXCEPTION_IN_PAGE_ERROR is raised, an error code comparing equal to errc::no_space_on_device will be returned. This may not always be the cause of the raised signal, but it is by far the most likely.
Memory Allocations
None if a QUICKCPPLIB_NAMESPACE::signal_guard_install is already instanced.
409 { return _mh.write(reqs, d); }
virtual io_result< const_buffers_type > write(io_request< const_buffers_type > reqs, deadline d=deadline()) noexcept override
Write data to the mapped view.

◆ zero()

virtual result<extent_type> afio_v2_xxx::mapped_file_handle::zero ( extent_type  offset,
extent_type  bytes,
deadline  d = deadline() 
)
inlineoverridevirtualnoexcept

Efficiently zero, and possibly deallocate, data on storage.

On most major operating systems and with recent filing systems which are "extents based", one can deallocate the physical storage of a file, causing the space deallocated to appear all bits zero. This call attempts to deallocate whole pages (usually 4Kb) entirely, and memset's any excess to all bits zero. This call works on most Linux filing systems with a recent kernel, Microsoft Windows with NTFS, and FreeBSD with ZFS. On other systems it simply writes zeros.

Returns
The bytes zeroed.
Parameters
offsetThe offset to start zeroing from.
bytesThe number of bytes to zero.
dAn optional deadline by which the i/o must complete, else it is cancelled. Note function may return significantly after this deadline if the i/o takes long to cancel.
Errors returnable
Any of the values POSIX write() can return, errc::timed_out, errc::operation_canceled. errc::not_supported may be returned if deadline i/o is not possible with this particular handle configuration (e.g. writing to regular files on POSIX or writing to a non-overlapped HANDLE on Windows).
Memory Allocations
The default synchronous implementation in file_handle performs no memory allocation. The asynchronous implementation in async_file_handle may perform one calloc and one free.

Reimplemented from afio_v2_xxx::file_handle.

371  {
372  OUTCOME_TRYV(_mh.zero_memory({_mh.address() + offset, bytes}));
373  return bytes;
374  }
result< void > zero_memory(buffer_type region) noexcept

The documentation for this class was generated from the following file: