AFIO  v2.00 late alpha
afio_v2_xxx::map_handle Class Referenceabstract

A handle to a memory mapped region of memory, either backed by the system page file or by a section. More...

#include "map_handle.hpp"

Inheritance diagram for afio_v2_xxx::map_handle:
afio_v2_xxx::io_handle afio_v2_xxx::handle

Public Types

using extent_type = io_handle::extent_type
 
using size_type = io_handle::size_type
 
using mode = io_handle::mode
 
using creation = io_handle::creation
 
using caching = io_handle::caching
 
using flag = io_handle::flag
 
using buffer_type = io_handle::buffer_type
 
using const_buffer_type = io_handle::const_buffer_type
 
using buffers_type = io_handle::buffers_type
 
using const_buffers_type = io_handle::const_buffers_type
 
template<class T >
using io_request = io_handle::io_request< T >
 
template<class T >
using io_result = io_handle::io_result< T >
 
using path_type = handle::path_type
 
enum  bitfield__flag : unsigned {
  none = 0, none = 2, none = 1, unlink_on_first_close = 1U << 0U,
  disable_safety_fsyncs = 1U << 2U, disable_safety_unlinks = 1U << 3U, disable_prefetching = 1U << 4U, maximum_prefetching = 1U << 5U,
  win_disable_unlink_emulation = 1U << 24U, win_disable_sparse_file_creation = 1U << 25U, overlapped = 1U << 28U, byte_lock_insanity = 1U << 29U,
  anonymous_inode = 1U << 30U
}
 Bitwise flags which can be specified. More...
 

Public Member Functions

constexpr map_handle ()
 Default constructor.
 
constexpr map_handle (map_handle &&o) noexcept
 Implicit move construction of map_handle permitted.
 
 map_handle (const map_handle &)=delete
 No copy construction (use clone())
 
map_handleoperator= (map_handle &&o) noexcept
 Move assignment of map_handle permitted.
 
map_handleoperator= (const map_handle &)=delete
 No copy assignment.
 
void swap (map_handle &o) noexcept
 Swap with another instance.
 
virtual result< void > close () noexcept override
 Unmap the mapped view.
 
virtual native_handle_type release () noexcept override
 Releases the mapped view, but does NOT release the native handle.
 
virtual io_result< const_buffers_typebarrier (io_request< const_buffers_type > reqs=io_request< const_buffers_type >(), bool wait_for_device=false, bool and_metadata=false, deadline d=deadline()) noexcept override
 
section_handlesection () const noexcept
 The memory section this handle is using.
 
void set_section (section_handle *s) noexcept
 Sets the memory section this handle is using.
 
byte * address () const noexcept
 The address in memory where this mapped view resides.
 
extent_type offset () const noexcept
 The offset of the memory map.
 
size_type capacity () const noexcept
 The reservation size of the memory map.
 
size_type length () const noexcept
 The size of the memory map. This is the accessible size, NOT the reservation size.
 
bool is_nvram () const noexcept
 True if the map is of non-volatile RAM.
 
result< size_typeupdate_map () noexcept
 Update the size of the memory map to that of any backing section, up to the reservation limit.
 
result< size_typetruncate (size_type newsize, bool permit_relocation=false) noexcept
 
result< buffer_typecommit (buffer_type region, section_handle::flag flag=section_handle::flag::readwrite) noexcept
 Ask the system to commit the system resources to make the memory represented by the buffer available with the given permissions. addr and length should be page aligned (see utils::page_sizes()), if not the returned buffer is the region actually committed.
 
result< buffer_typedecommit (buffer_type region) noexcept
 Ask the system to make the memory represented by the buffer unavailable and to decommit the system resources representing them. addr and length should be page aligned (see utils::page_sizes()), if not the returned buffer is the region actually decommitted.
 
result< void > zero_memory (buffer_type region) noexcept
 
result< buffer_typedo_not_store (buffer_type region) noexcept
 
virtual io_result< buffers_typeread (io_request< buffers_type > reqs, deadline d=deadline()) noexcept override
 Read data from the mapped view. More...
 
virtual io_result< const_buffers_typewrite (io_request< const_buffers_type > reqs, deadline d=deadline()) noexcept override
 Write data to the mapped view. More...
 
virtual size_t max_buffers () const noexcept
 The maximum number of buffers which a single read or write syscall can process at a time for this specific open handle. On POSIX, this is known as IOV_MAX. More...
 
virtual io_result< buffers_typeread (io_request< buffers_type > reqs, deadline d=deadline()) noexcept
 Read data from the open handle. More...
 
io_result< buffers_typeread (extent_type offset, std::initializer_list< buffer_type > lst, deadline d=deadline()) noexcept
 
virtual io_result< const_buffers_typewrite (io_request< const_buffers_type > reqs, deadline d=deadline()) noexcept
 Write data to the open handle. More...
 
io_result< const_buffers_typewrite (extent_type offset, std::initializer_list< const_buffer_type > lst, deadline d=deadline()) noexcept
 
virtual io_result< const_buffers_typebarrier (io_request< const_buffers_type > reqs=io_request< const_buffers_type >(), bool wait_for_device=false, bool and_metadata=false, deadline d=deadline()) noexcept=0
 Issue a write reordering barrier such that writes preceding the barrier will reach storage before writes after this barrier. More...
 
virtual result< extent_guardlock (extent_type offset, extent_type bytes, bool exclusive=true, deadline d=deadline()) noexcept
 Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes through the same semantics as the underlying OS call, including any POSIX insanity present on your platform: More...
 
result< extent_guardlock (io_request< buffers_type > reqs, deadline d=deadline()) noexcept
 
result< extent_guardlock (io_request< const_buffers_type > reqs, deadline d=deadline()) noexcept
 
result< extent_guardtry_lock (extent_type offset, extent_type bytes, bool exclusive=true) noexcept
 
virtual void unlock (extent_type offset, extent_type bytes) noexcept
 Unlocks a byte range previously locked. More...
 
void swap (handle &o) noexcept
 Swap with another instance.
 
virtual result< path_typecurrent_path () const noexcept
 
result< handleclone () const noexcept
 
bool is_valid () const noexcept
 True if the handle is valid (and usually open)
 
bool is_readable () const noexcept
 True if the handle is readable.
 
bool is_writable () const noexcept
 True if the handle is writable.
 
bool is_append_only () const noexcept
 True if the handle is append only.
 
virtual result< void > set_append_only (bool enable) noexcept
 
bool is_overlapped () const noexcept
 True if overlapped.
 
bool is_seekable () const noexcept
 True if seekable.
 
bool requires_aligned_io () const noexcept
 True if requires aligned i/o.
 
bool is_regular () const noexcept
 True if a regular file or device.
 
bool is_directory () const noexcept
 True if a directory.
 
bool is_symlink () const noexcept
 True if a symlink.
 
bool is_multiplexer () const noexcept
 True if a multiplexer like BSD kqueues, Linux epoll or Windows IOCP.
 
bool is_process () const noexcept
 True if a process.
 
bool is_section () const noexcept
 True if a memory section.
 
caching kernel_caching () const noexcept
 Kernel cache strategy used by this handle.
 
bool are_reads_from_cache () const noexcept
 True if the handle uses the kernel page cache for reads.
 
bool are_writes_durable () const noexcept
 True if writes are safely on storage on completion.
 
bool are_safety_fsyncs_issued () const noexcept
 True if issuing safety fsyncs is on.
 
flag flags () const noexcept
 The flags this handle was opened with.
 
native_handle_type native_handle () const noexcept
 The native handle used by this handle.
 

Static Public Member Functions

static const_buffer_type barrier (const_buffer_type req, bool evict=false) noexcept
 
static result< map_handlemap (size_type bytes, section_handle::flag _flag=section_handle::flag::readwrite) noexcept
 
static result< map_handlemap (section_handle &section, size_type bytes=0, extent_type offset=0, section_handle::flag _flag=section_handle::flag::readwrite) noexcept
 
static result< span< buffer_type > > prefetch (span< buffer_type > regions) noexcept
 Ask the system to begin to asynchronously prefetch the span of memory regions given, returning the regions actually prefetched. Note that on Windows 7 or earlier the system call to implement this was not available, and so you will see an empty span returned.
 
static result< buffer_typeprefetch (buffer_type region) noexcept
 

Protected Member Functions

 map_handle (section_handle *section)
 

Protected Attributes

section_handle_section {nullptr}
 
byte * _addr {nullptr}
 
extent_type _offset {0}
 
size_type _reservation {0}
 
size_type _length {0}
 
section_handle::flag _flag {section_handle::flag::none}
 
caching _caching {caching::none}
 
flag _flags {flag::none}
 
native_handle_type _v
 

Friends

class mapped_file_handle
 

Detailed Description

A handle to a memory mapped region of memory, either backed by the system page file or by a section.

An important concept to realise with mapped regions is that they can far exceed the size of their backing storage. This allows one to reserve address space for a file which may grow in the future. This is how mapped_file_handle is implemented to provide very fast memory mapped file i/o of a potentially growing file.

The size you specify when creating the map handle is the address space reservation. The map's length() will return the last known valid length of the mapped data i.e. the backing storage's length at the time of construction. This length is used by read() and write() to prevent reading and writing off the end of the mapped region. You can update this length to the backing storage's length using update_map() up to the reservation limit.

You can attempt to modify the address space reservation after creation using truncate(). If successful, this will be more efficient than tearing down the map and creating a new larger map.

The native handle returned by this map handle is always that of the backing storage, but closing this handle does not close that of the backing storage, nor does releasing this handle release that of the backing storage. Locking byte ranges of this handle is therefore equal to locking byte ranges in the original backing storage, which can be very useful.

Barriers:

map_handle, because it implements io_handle, implements barrier() in a very conservative way to account for OS differences i.e. it calls msync(), and then the barrier() implementation for the backing file (probably fsync() or equivalent on most platforms, which synchronises the entire file).

This is vast overkill if you are using non-volatile RAM, so a special inlined barrier() implementation taking a single buffer and no other arguments is also provided. This calls the appropriate architecture-specific instructions to cause the CPU to write all preceding writes out of the write buffers and CPU caches to main memory, so for Intel CPUs this would be CLWB <each cache line>; SFENCE;. As this is inlined, it ought to produce optimal code. If your CPU does not support the requisite instructions (or AFIO has not added support), and empty buffer will be returned to indicate that nothing was barriered, same as the normal barrier() function.

See also
mapped_file_handle, algorithm::mapped_span

Member Enumeration Documentation

◆ bitfield__flag

enum afio_v2_xxx::handle::bitfield__flag : unsigned
inherited

Bitwise flags which can be specified.

Enumerator
none 

No flags.

none 

No ability to read or write anything, but can synchronise (SYNCHRONIZE or 0)

none 

No caching whatsoever, all reads and writes come from storage (i.e. O_DIRECT|O_SYNC). Align all i/o to 4Kb boundaries for this to work. flag_disable_safety_fsyncs can be used here.

unlink_on_first_close 

Unlinks the file on handle close. On POSIX, this simply unlinks whatever is pointed to by path() upon the call of close() if and only if the inode matches. On Windows, if you are on Windows 10 1709 or later, exactly the same thing occurs. If on previous editions of Windows, the file entry does not disappears but becomes unavailable for anyone else to open with an errc::resource_unavailable_try_again error return. Because this is confusing, unless the win_disable_unlink_emulation flag is also specified, this POSIX behaviour is somewhat emulated by AFIO on older Windows by renaming the file to a random name on close() causing it to appear to have been unlinked immediately.

disable_safety_fsyncs 

Some kernel caching modes have unhelpfully inconsistent behaviours in getting your data onto storage, so by default unless this flag is specified AFIO adds extra fsyncs to the following operations for the caching modes specified below: truncation of file length either explicitly or during file open. closing of the handle either explicitly or in the destructor.

Additionally on Linux only to prevent loss of file metadata: On the parent directory whenever a file might have been created. On the parent directory on file close.

This only occurs for these kernel caching modes: caching::none caching::reads caching::reads_and_metadata caching::safety_fsyncs

disable_safety_unlinks 

file_handle::unlink() could accidentally delete the wrong file if someone has renamed the open file handle since the time it was opened. To prevent this occuring, where the OS doesn't provide race free unlink-by-open-handle we compare the inode of the path we are about to unlink with that of the open handle before unlinking.

Warning
This does not prevent races where in between the time of checking the inode and executing the unlink a third party changes the item about to be unlinked. Only operating systems with a true race-free unlink syscall are race free.
disable_prefetching 

Ask the OS to disable prefetching of data. This can improve random i/o performance.

maximum_prefetching 

Ask the OS to maximise prefetching of data, possibly prefetching the entire file into kernel cache. This can improve sequential i/o performance.

win_disable_unlink_emulation 

See the documentation for unlink_on_first_close

win_disable_sparse_file_creation 

Microsoft Windows NTFS, having been created in the late 1980s, did not originally implement extents-based storage and thus could only represent sparse files via efficient compression of intermediate zeros. With NTFS v3.0 (Microsoft Windows 2000), a proper extents-based on-storage representation was added, thus allowing only 64Kb extent chunks written to be stored irrespective of whatever the maximum file extent was set to.

For various historical reasons, extents-based storage is disabled by default in newly created files on NTFS, unlike in almost every other major filing system. You have to explicitly "opt in" to extents-based storage.

As extents-based storage is nearly cost free on NTFS, AFIO by default opts in to extents-based storage for any empty file it creates. If you don't want this, you can specify this flag to prevent that happening.

overlapped 

On Windows, create any new handles with OVERLAPPED semantics.

byte_lock_insanity 

Using insane POSIX byte range locks.

anonymous_inode 

This is an inode created with no representation on the filing system.

97  {
98  none = 0, //!< No flags
99  /*! Unlinks the file on handle close. On POSIX, this simply unlinks whatever is pointed
100  to by `path()` upon the call of `close()` if and only if the inode matches. On Windows,
101  if you are on Windows 10 1709 or later, exactly the same thing occurs. If on previous
102  editions of Windows, the file entry does not disappears but becomes unavailable for
103  anyone else to open with an `errc::resource_unavailable_try_again` error return. Because this is confusing, unless the
104  `win_disable_unlink_emulation` flag is also specified, this POSIX behaviour is
105  somewhat emulated by AFIO on older Windows by renaming the file to a random name on `close()`
106  causing it to appear to have been unlinked immediately.
107  */
108  unlink_on_first_close = 1U << 0U,
109 
110  /*! Some kernel caching modes have unhelpfully inconsistent behaviours
111  in getting your data onto storage, so by default unless this flag is
112  specified AFIO adds extra fsyncs to the following operations for the
113  caching modes specified below:
114  * truncation of file length either explicitly or during file open.
115  * closing of the handle either explicitly or in the destructor.
116 
117  Additionally on Linux only to prevent loss of file metadata:
118  * On the parent directory whenever a file might have been created.
119  * On the parent directory on file close.
120 
121  This only occurs for these kernel caching modes:
122  * caching::none
123  * caching::reads
124  * caching::reads_and_metadata
125  * caching::safety_fsyncs
126  */
127  disable_safety_fsyncs = 1U << 2U,
128  /*! `file_handle::unlink()` could accidentally delete the wrong file if someone has
129  renamed the open file handle since the time it was opened. To prevent this occuring,
130  where the OS doesn't provide race free unlink-by-open-handle we compare the inode of
131  the path we are about to unlink with that of the open handle before unlinking.
132  \warning This does not prevent races where in between the time of checking the inode
133  and executing the unlink a third party changes the item about to be unlinked. Only
134  operating systems with a true race-free unlink syscall are race free.
135  */
136  disable_safety_unlinks = 1U << 3U,
137  /*! Ask the OS to disable prefetching of data. This can improve random
138  i/o performance.
139  */
140  disable_prefetching = 1U << 4U,
141  /*! Ask the OS to maximise prefetching of data, possibly prefetching the entire file
142  into kernel cache. This can improve sequential i/o performance.
143  */
144  maximum_prefetching = 1U << 5U,
145 
146  win_disable_unlink_emulation = 1U << 24U, //!< See the documentation for `unlink_on_first_close`
147  /*! Microsoft Windows NTFS, having been created in the late 1980s, did not originally
148  implement extents-based storage and thus could only represent sparse files via
149  efficient compression of intermediate zeros. With NTFS v3.0 (Microsoft Windows 2000),
150  a proper extents-based on-storage representation was added, thus allowing only 64Kb
151  extent chunks written to be stored irrespective of whatever the maximum file extent
152  was set to.
153 
154  For various historical reasons, extents-based storage is disabled by default in newly
155  created files on NTFS, unlike in almost every other major filing system. You have to
156  explicitly "opt in" to extents-based storage.
157 
158  As extents-based storage is nearly cost free on NTFS, AFIO by default opts in to
159  extents-based storage for any empty file it creates. If you don't want this, you
160  can specify this flag to prevent that happening.
161  */
163 
164  // NOTE: IF UPDATING THIS UPDATE THE std::ostream PRINTER BELOW!!!
165 
166  overlapped = 1U << 28U, //!< On Windows, create any new handles with OVERLAPPED semantics
167  byte_lock_insanity = 1U << 29U, //!< Using insane POSIX byte range locks
168  anonymous_inode = 1U << 30U //!< This is an inode created with no representation on the filing system
169  }
See the documentation for unlink_on_first_close
Definition: handle.hpp:146
This is an inode created with no representation on the filing system.
Definition: handle.hpp:168
On Windows, create any new handles with OVERLAPPED semantics.
Definition: handle.hpp:166
No flags.
Definition: handle.hpp:98
Using insane POSIX byte range locks.
Definition: handle.hpp:167

Member Function Documentation

◆ barrier() [1/2]

virtual io_result<const_buffers_type> afio_v2_xxx::io_handle::barrier ( io_request< const_buffers_type reqs = io_requestconst_buffers_type >(),
bool  wait_for_device = false,
bool  and_metadata = false,
deadline  d = deadline() 
)
pure virtualnoexceptinherited

Issue a write reordering barrier such that writes preceding the barrier will reach storage before writes after this barrier.

Warning
Assume that this call is a no-op. It is not reliably implemented in many common use cases, for example if your code is running inside a LXC container, or if the user has mounted the filing system with non-default options. Instead open the handle with caching::reads which means that all writes form a strict sequential order not completing until acknowledged by the storage device. Filing system can and do use different algorithms to give much better performance with caching::reads, some (e.g. ZFS) spectacularly better.
Let me repeat again: consider this call to be a hint to poke the kernel with a stick to go start to do some work sooner rather than later. It may be ignored entirely.
For portability, you can only assume that barriers write order for a single handle instance. You cannot assume that barriers write order across multiple handles to the same inode, or across processes.
Returns
The buffers barriered, which may not be the buffers input. The size of each scatter-gather buffer is updated with the number of bytes of that buffer barriered.
Parameters
reqsA scatter-gather and offset request for what range to barrier. May be ignored on some platforms which always write barrier the entire file. Supplying a default initialised reqs write barriers the entire file.
wait_for_deviceTrue if you want the call to wait until data reaches storage and that storage has acknowledged the data is physically written. Slow.
and_metadataTrue if you want the call to sync the metadata for retrieving the writes before the barrier after a sudden power loss event. Slow. Setting this to false enables much faster performance, especially on non-volatile memory.
dAn optional deadline by which the i/o must complete, else it is cancelled. Note function may return significantly after this deadline if the i/o takes long to cancel.
Errors returnable
Any of the values POSIX fdatasync() or Windows NtFlushBuffersFileEx() can return.
Memory Allocations
None.

◆ barrier() [2/2]

static const_buffer_type afio_v2_xxx::map_handle::barrier ( const_buffer_type  req,
bool  evict = false 
)
inlinestaticnoexcept

Lightweight inlined barrier which causes the CPU to write out all buffered writes and dirty cache lines in the request to main memory.

Returns
The cache lines actually barriered. This may be empty. This function does not return an error.
Parameters
reqThe range of cache lines to write barrier.
evictWhether to also evict the cache lines from CPU caches, useful if they will not be used again.

Upon return, one knows that memory in the returned buffer has been barriered (it may be empty if there is no support for this operation in AFIO, or if the current CPU does not support this operation). You may find the is_nvram() observer of particular use here.

364  {
365  const_buffer_type ret{(const_buffer_type::pointer)(((uintptr_t) req.data) & 31), 0};
366  ret.len = req.data + req.len - ret.data;
367  for(const_buffer_type::pointer addr = ret.data; addr < ret.data + ret.len; addr += 32)
368  {
369  // Slightly UB ...
370  auto *p = reinterpret_cast<const persistent<byte> *>(addr);
371  if(memory_flush_none == p->flush(evict ? memory_flush_evict : memory_flush_retain))
372  {
373  req.len = 0;
374  break;
375  }
376  }
377  return ret;
378  }
const byte * pointer
Type of the pointer to memory.
Definition: io_handle.hpp:70
size_type len
The number of bytes to write from this address. Try to make this a 64 byte multiple, or ideally, a whole multiple of page_size().
Definition: io_handle.hpp:77

◆ clone()

result<handle> afio_v2_xxx::handle::clone ( ) const
inlinenoexceptinherited

Clone this handle (copy constructor is disabled to avoid accidental copying)

Errors returnable
Any of the values POSIX dup() or DuplicateHandle() can return.

◆ current_path()

virtual result<path_type> afio_v2_xxx::handle::current_path ( ) const
inlinevirtualnoexceptinherited

Returns the current path of the open handle as said by the operating system. Note that you are NOT guaranteed that any path refreshed bears any resemblance to the original, some operating systems will return some different path which still reaches the same inode via some other route e.g. hardlinks, dereferenced symbolic links, etc. Windows and Linux correctly track changes to the specific path the handle was opened with, not getting confused by other hard links. MacOS nearly gets it right, but under some circumstances e.g. renaming may switch to a different hard link's path which is almost certainly a bug.

If AFIO was not able to determine the current path for this open handle e.g. the inode has been unlinked, it returns an empty path. Be aware that FreeBSD can return an empty (deleted) path for file inodes no longer cached by the kernel path cache, AFIO cannot detect the difference. FreeBSD will also return any path leading to the inode if it is hard linked. FreeBSD does implement path retrieval for directory inodes correctly however, and see algorithm::stablized_path<T> for a handle adapter which makes use of that.

On Linux if /proc is not mounted, this call fails with an error. All APIs in AFIO which require the use of current_path() can be told to not use it e.g. flag::disable_safety_unlinks. It is up to you to detect if current_path() is not working, and to change how you call AFIO appropriately.

Warning
This call is expensive, it always asks the kernel for the current path, and no checking is done to ensure what the kernel returns is accurate or even sensible. Be aware that despite these precautions, paths are unstable and can change randomly at any moment. Most code written to use absolute file systems paths is racy, so don't do it, use path_handle to fix a base location on the file system and work from that anchor instead!
Memory Allocations
At least one malloc for the path_type, likely several more.
See also
algorithm::cached_parent_handle_adapter<T> which overrides this with an implementation based on retrieving the current path of a cached handle to the parent directory. On platforms with instability or failure to retrieve the correct current path for regular files, the cached parent handle adapter works around the problem by taking advantage of directory inodes not having the same instability problems on any platform.

◆ do_not_store()

result<buffer_type> afio_v2_xxx::map_handle::do_not_store ( buffer_type  region)
inlinenoexcept

Ask the system to unset the dirty flag for the memory represented by the buffer. This will prevent any changes not yet sent to the backing storage from being sent in the future, also if the system kicks out this page and reloads it you may see some edition of the underlying storage instead of what was here. addr and length should be page aligned (see utils::page_sizes()), if not the returned buffer is the region actually undirtied.

Warning
This function destroys the contents of unwritten pages in the region in a totally unpredictable fashion. Only use it if you don't care how much of the region reaches physical storage or not. Note that the region is not necessarily zeroed, and may be randomly zeroed.
Note
Microsoft Windows does not support unsetting the dirty flag on file backed maps, so on Windows this call does nothing.

◆ lock() [1/3]

virtual result<extent_guard> afio_v2_xxx::io_handle::lock ( extent_type  offset,
extent_type  bytes,
bool  exclusive = true,
deadline  d = deadline() 
)
inlinevirtualnoexceptinherited

Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes through the same semantics as the underlying OS call, including any POSIX insanity present on your platform:

  • Any fd closed on an inode must release all byte range locks on that inode for all other fds. If your OS isn't new enough to support the non-insane lock API, flag::byte_lock_insanity will be set in flags() after the first call to this function.
  • Threads replace each other's locks, indeed locks replace each other's locks.

You almost cetainly should use your choice of an algorithm::shared_fs_mutex::* instead of this as those are more portable and performant.

Warning
This is a low-level API which you should not use directly in portable code. Another issue is that atomic lock upgrade/downgrade, if your platform implements that (you should assume it does not in portable code), means that on POSIX you need to release the old extent_guard after creating a new one over the same byte range, otherwise the old extent_guard's destructor will simply unlock the range entirely. On Windows however upgrade/downgrade locks overlay, so on that platform you must not release the old extent_guard. Look into algorithm::shared_fs_mutex::safe_byte_ranges for a portable solution.
Returns
An extent guard, the destruction of which will call unlock().
Parameters
offsetThe offset to lock. Note that on POSIX the top bit is always cleared before use as POSIX uses signed transport for offsets. If you want an advisory rather than mandatory lock on Windows, one technique is to force top bit set so the region you lock is not the one you will i/o - obviously this reduces maximum file size to (2^63)-1.
bytesThe number of bytes to lock. Zero means lock the entire file using any more efficient alternative algorithm where available on your platform (specifically, on BSD and OS X use flock() for non-insane semantics).
exclusiveWhether the lock is to be exclusive.
dAn optional deadline by which the lock must complete, else it is cancelled.
Errors returnable
Any of the values POSIX fcntl() can return, errc::timed_out, errc::not_supported may be returned if deadline i/o is not possible with this particular handle configuration (e.g. non-overlapped HANDLE on Windows).
Memory Allocations
The default synchronous implementation in file_handle performs no memory allocation. The asynchronous implementation in async_file_handle performs one calloc and one free.

◆ lock() [2/3]

result<extent_guard> afio_v2_xxx::io_handle::lock ( io_request< buffers_type reqs,
deadline  d = deadline() 
)
inlinenoexceptinherited

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

424  {
425  size_t bytes = 0;
426  for(auto &i : reqs.buffers)
427  {
428  if(bytes + i.len < bytes)
429  {
430  return errc::value_too_large;
431  }
432  bytes += i.len;
433  }
434  return lock(reqs.offset, bytes, false, d);
435  }
virtual result< extent_guard > lock(extent_type offset, extent_type bytes, bool exclusive=true, deadline d=deadline()) noexcept
Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes throu...

◆ lock() [3/3]

result<extent_guard> afio_v2_xxx::io_handle::lock ( io_request< const_buffers_type reqs,
deadline  d = deadline() 
)
inlinenoexceptinherited

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

438  {
439  size_t bytes = 0;
440  for(auto &i : reqs.buffers)
441  {
442  if(bytes + i.len < bytes)
443  {
444  return errc::value_too_large;
445  }
446  bytes += i.len;
447  }
448  return lock(reqs.offset, bytes, true, d);
449  }
virtual result< extent_guard > lock(extent_type offset, extent_type bytes, bool exclusive=true, deadline d=deadline()) noexcept
Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes throu...

◆ map() [1/2]

static result<map_handle> afio_v2_xxx::map_handle::map ( size_type  bytes,
section_handle::flag  _flag = section_handle::flag::readwrite 
)
inlinestaticnoexcept

Create new memory and map it into view.

Parameters
bytesHow many bytes to create and map. Typically will be rounded up to a multiple of the page size (see utils::page_sizes()) on POSIX, 64Kb on Windows.
_flagThe permissions with which to map the view. flag::none can be useful for reserving virtual address space without committing system resources, use commit() to later change availability of memory.
Note
On Microsoft Windows this constructor uses the faster VirtualAlloc() which creates less versatile page backed memory. If you want anonymous memory allocated from a paging file backed section instead, create a page file backed section and then a mapped view from that using the other constructor. This makes available all those very useful VM tricks Windows can do with section mapped memory which VirtualAlloc() memory cannot do.
Errors returnable
Any of the values POSIX mmap() or VirtualAlloc() can return.

◆ map() [2/2]

static result<map_handle> afio_v2_xxx::map_handle::map ( section_handle section,
size_type  bytes = 0,
extent_type  offset = 0,
section_handle::flag  _flag = section_handle::flag::readwrite 
)
inlinestaticnoexcept

Create a memory mapped view of a backing storage, optionally reserving additional address space for later growth.

Parameters
sectionA memory section handle specifying the backing storage to use.
bytesHow many bytes to reserve (0 = the size of the section). Rounded up to nearest 64Kb on Windows.
offsetThe offset into the backing storage to map from. Typically needs to be at least a multiple of the page size (see utils::page_sizes()), on Windows it needs to be a multiple of the kernel memory allocation granularity (typically 64Kb).
_flagThe permissions with which to map the view which are constrained by the permissions of the memory section. flag::none can be useful for reserving virtual address space without committing system resources, use commit() to later change availability of memory.
Errors returnable
Any of the values POSIX mmap() or NtMapViewOfSection() can return.

◆ max_buffers()

virtual size_t afio_v2_xxx::io_handle::max_buffers ( ) const
inlinevirtualnoexceptinherited

The maximum number of buffers which a single read or write syscall can process at a time for this specific open handle. On POSIX, this is known as IOV_MAX.

Note that the actual number of buffers accepted for a read or a write may be significantly lower than this system-defined limit, depending on available resources. The read() or write() call will return the buffers accepted.

Note also that some OSs will error out if you supply more than this limit to read() or write(), but other OSs do not. Some OSs guarantee that each i/o syscall has effects atomically visible or not to other i/o, other OSs do not.

Microsoft Windows and OS X does not implement scatter-gather file i/o syscalls. Thus this function will always return 1 in that situation.

◆ prefetch()

static result<buffer_type> afio_v2_xxx::map_handle::prefetch ( buffer_type  region)
inlinestaticnoexcept

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

500  {
501  OUTCOME_TRY(ret, prefetch(span<buffer_type>(&region, 1)));
502  return *ret.data();
503  }
static result< span< buffer_type > > prefetch(span< buffer_type > regions) noexcept
Ask the system to begin to asynchronously prefetch the span of memory regions given, returning the regions actually prefetched. Note that on Windows 7 or earlier the system call to implement this was not available, and so you will see an empty span returned.

◆ read() [1/3]

virtual io_result<buffers_type> afio_v2_xxx::io_handle::read ( io_request< buffers_type reqs,
deadline  d = deadline() 
)
inlinevirtualnoexceptinherited

Read data from the open handle.

Warning
Depending on the implementation backend, very different buffers may be returned than you supplied. You should always use the buffers returned and assume that they point to different memory and that each buffer's size will have changed.
Returns
The buffers read, which may not be the buffers input. The size of each scatter-gather buffer is updated with the number of bytes of that buffer transferred, and the pointer to the data may be completely different to what was submitted (e.g. it may point into a memory map).
Parameters
reqsA scatter-gather and offset request.
dAn optional deadline by which the i/o must complete, else it is cancelled. Note function may return significantly after this deadline if the i/o takes long to cancel.
Errors returnable
Any of the values POSIX read() can return, errc::timed_out, errc::operation_canceled. errc::not_supported may be returned if deadline i/o is not possible with this particular handle configuration (e.g. reading from regular files on POSIX or reading from a non-overlapped HANDLE on Windows).
Memory Allocations
The default synchronous implementation in file_handle performs no memory allocation. The asynchronous implementation in async_file_handle performs one calloc and one free.

◆ read() [2/3]

io_result<buffers_type> afio_v2_xxx::io_handle::read ( extent_type  offset,
std::initializer_list< buffer_type lst,
deadline  d = deadline() 
)
inlinenoexceptinherited

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

236  {
237  buffer_type *_reqs = reinterpret_cast<buffer_type *>(alloca(sizeof(buffer_type) * lst.size()));
238  memcpy(_reqs, lst.begin(), sizeof(buffer_type) * lst.size());
239  io_request<buffers_type> reqs(buffers_type(_reqs, lst.size()), offset);
240  return read(reqs, d);
241  }
span< buffer_type > buffers_type
The scatter buffers type used by this handle. Guaranteed to be TrivialType apart from construction...
Definition: io_handle.hpp:86
virtual io_result< buffers_type > read(io_request< buffers_type > reqs, deadline d=deadline()) noexcept
Read data from the open handle.

◆ read() [3/3]

virtual io_result<buffers_type> afio_v2_xxx::map_handle::read ( io_request< buffers_type reqs,
deadline  d = deadline() 
)
inlineoverridevirtualnoexcept

Read data from the mapped view.

Note
Because this implementation never copies memory, you can pass in buffers with a null address. As this function never reads any memory, no attempt to trap signal raises can be made, this falls onto the user of this function. See QUICKCPPLIB_NAMESPACE::signal_guard for a helper function.
Returns
The buffers read, which will never be the buffers input, because they will point into the mapped view. The size of each scatter-gather buffer is updated with the number of bytes of that buffer transferred.
Parameters
reqsA scatter-gather and offset request.
dIgnored.
Errors returnable
None, though the various signals and structured exception throws common to using memory maps may occur.
Memory Allocations
None.

◆ set_append_only()

virtual result<void> afio_v2_xxx::handle::set_append_only ( bool  enable)
inlinevirtualnoexceptinherited

Changes whether this handle is append only or not.

Warning
On Windows this is implemented as a bit of a hack to make it fast like on POSIX, so make sure you open the handle for read/write originally. Note unlike on POSIX the append_only disposition will be the only one toggled, seekable and readable will remain turned on.
Errors returnable
Whatever POSIX fcntl() returns. On Windows nothing is changed on the handle.
Memory Allocations
No memory allocation.

◆ truncate()

result<size_type> afio_v2_xxx::map_handle::truncate ( size_type  newsize,
bool  permit_relocation = false 
)
inlinenoexcept

Resize the reservation of the memory map without changing the address (unless the map was zero sized, in which case a new address will be chosen).

If shrinking, address space is released on POSIX, and on Windows if the new size is zero. If the new size is zero, the address is set to null to prevent surprises. Windows does not support modifying existing mapped regions, so if the new size is not zero, the call will probably fail. Windows should let you truncate a previous extension however, if it is exact.

If expanding, an attempt is made to map in new reservation immediately after the current address reservation, thus extending the reservation. If anything else is mapped in after the current reservation, the function fails.

Note
On all supported platforms apart from OS X, proprietary flags exist to avoid performing a map if a map extension cannot be immediately placed after the current map. On OS X, we hint where we'd like the new map to go, but if something is already there OS X will place the map elsewhere. In this situation, we delete the new map and return failure, which is inefficient, but there is nothing else we can do.
Returns
The bytes actually reserved.
Parameters
newsizeThe bytes to truncate the map reservation to. Rounded up to the nearest page size (POSIX) or 64Kb on Windows.
permit_relocationPermit the address to change (some OSs provide a syscall for resizing a memory map).
Errors returnable
Any of the values POSIX mremap(), mmap(addr) or VirtualAlloc(addr) can return.

◆ try_lock()

result<extent_guard> afio_v2_xxx::io_handle::try_lock ( extent_type  offset,
extent_type  bytes,
bool  exclusive = true 
)
inlinenoexceptinherited

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

421 { return lock(offset, bytes, exclusive, deadline(std::chrono::seconds(0))); }
virtual result< extent_guard > lock(extent_type offset, extent_type bytes, bool exclusive=true, deadline d=deadline()) noexcept
Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes throu...

◆ unlock()

virtual void afio_v2_xxx::io_handle::unlock ( extent_type  offset,
extent_type  bytes 
)
inlinevirtualnoexceptinherited

Unlocks a byte range previously locked.

Parameters
offsetThe offset to unlock. This should be an offset previously locked.
bytesThe number of bytes to unlock. This should be a byte extent previously locked.
Errors returnable
Any of the values POSIX fcntl() can return.
Memory Allocations
None.

◆ write() [1/3]

virtual io_result<const_buffers_type> afio_v2_xxx::io_handle::write ( io_request< const_buffers_type reqs,
deadline  d = deadline() 
)
inlinevirtualnoexceptinherited

Write data to the open handle.

Warning
Depending on the implementation backend, not all of the buffers input may be written and the some buffers at the end of the returned buffers may return with zero bytes written. For example, with a zeroed deadline, some backends may only consume as many buffers as the system has available write slots for, thus for those backends this call is "non-blocking" in the sense that it will return immediately even if it could not schedule a single buffer write. Another example is that some implementations will not auto-extend the length of a file when a write exceeds the maximum extent, you will need to issue a truncate(newsize) first.
Returns
The buffers written, which may not be the buffers input. The size of each scatter-gather buffer is updated with the number of bytes of that buffer transferred.
Parameters
reqsA scatter-gather and offset request.
dAn optional deadline by which the i/o must complete, else it is cancelled. Note function may return significantly after this deadline if the i/o takes long to cancel.
Errors returnable
Any of the values POSIX write() can return, errc::timed_out, errc::operation_canceled. errc::not_supported may be returned if deadline i/o is not possible with this particular handle configuration (e.g. writing to regular files on POSIX or writing to a non-overlapped HANDLE on Windows).
Memory Allocations
The default synchronous implementation in file_handle performs no memory allocation. The asynchronous implementation in async_file_handle performs one calloc and one free.

◆ write() [2/3]

io_result<const_buffers_type> afio_v2_xxx::io_handle::write ( extent_type  offset,
std::initializer_list< const_buffer_type lst,
deadline  d = deadline() 
)
inlinenoexceptinherited

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

269  {
270  const_buffer_type *_reqs = reinterpret_cast<const_buffer_type *>(alloca(sizeof(const_buffer_type) * lst.size()));
271  memcpy(_reqs, lst.begin(), sizeof(const_buffer_type) * lst.size());
272  io_request<const_buffers_type> reqs(const_buffers_type(_reqs, lst.size()), offset);
273  return write(reqs, d);
274  }
span< const_buffer_type > const_buffers_type
The gather buffers type used by this handle. Guaranteed to be TrivialType apart from construction...
Definition: io_handle.hpp:88
virtual io_result< const_buffers_type > write(io_request< const_buffers_type > reqs, deadline d=deadline()) noexcept
Write data to the open handle.

◆ write() [3/3]

virtual io_result<const_buffers_type> afio_v2_xxx::map_handle::write ( io_request< const_buffers_type reqs,
deadline  d = deadline() 
)
inlineoverridevirtualnoexcept

Write data to the mapped view.

Note
This call traps signals and structured exception throws using QUICKCPPLIB_NAMESPACE::signal_guard. Instantiating a QUICKCPPLIB_NAMESPACE::signal_guard_install somewhere much higher up in the call stack will improve performance enormously. The signal guard may cost less than 100 CPU cycles depending on how you configure it. If you don't want the guard, you can write memory directly using address().
Returns
The buffers written, which will never be the buffers input because they will point at where the data was copied into the mapped view. The size of each scatter-gather buffer is updated with the number of bytes of that buffer transferred.
Parameters
reqsA scatter-gather and offset request.
dIgnored.
Errors returnable
If during the attempt to write the buffers to the map a SIGBUS or EXCEPTION_IN_PAGE_ERROR is raised, an error code comparing equal to errc::no_space_on_device will be returned. This may not always be the cause of the raised signal, but it is by far the most likely.
Memory Allocations
None if a QUICKCPPLIB_NAMESPACE::signal_guard_install is already instanced.

◆ zero_memory()

result<void> afio_v2_xxx::map_handle::zero_memory ( buffer_type  region)
inlinenoexcept

Zero the memory represented by the buffer. Differs from zero() because it acts on mapped memory, but may call zero() internally.

On Linux, Windows and FreeBSD any full 4Kb pages will be deallocated from the system entirely, including the extents for them in any backing storage. On newer Linux kernels the kernel can additionally swap whole 4Kb pages for freshly zeroed ones making this a very efficient way of zeroing large ranges of memory.

Errors returnable
Any of the errors returnable by madvise() or DiscardVirtualMemory or the zero() function.

The documentation for this class was generated from the following file: