AFIO  v2.00 late alpha
afio_v2_xxx::io_handle Class Referenceabstract

A handle to something capable of scatter-gather i/o. More...

#include "io_handle.hpp"

Inheritance diagram for afio_v2_xxx::io_handle:
afio_v2_xxx::handle afio_v2_xxx::file_handle afio_v2_xxx::map_handle afio_v2_xxx::async_file_handle afio_v2_xxx::mapped_file_handle

Classes

struct  buffer_type
 The scatter buffer type used by this handle. Guaranteed to be TrivialType and StandardLayoutType. More...
 
struct  const_buffer_type
 The gather buffer type used by this handle. Guaranteed to be TrivialType and StandardLayoutType. More...
 
class  extent_guard
 RAII holder a locked extent of bytes in a file. More...
 
struct  io_request
 The i/o request type used by this handle. Guaranteed to be TrivialType apart from construction, and StandardLayoutType. More...
 
struct  io_result
 The i/o result type used by this handle. Guaranteed to be TrivialType apart from construction.. More...
 

Public Types

using path_type = handle::path_type
 
using extent_type = handle::extent_type
 
using size_type = handle::size_type
 
using mode = handle::mode
 
using creation = handle::creation
 
using caching = handle::caching
 
using flag = handle::flag
 
using buffers_type = span< buffer_type >
 The scatter buffers type used by this handle. Guaranteed to be TrivialType apart from construction, and StandardLayoutType.
 
using const_buffers_type = span< const_buffer_type >
 The gather buffers type used by this handle. Guaranteed to be TrivialType apart from construction, and StandardLayoutType.
 

Public Member Functions

constexpr io_handle ()=default
 Default constructor. More...
 
constexpr io_handle (native_handle_type h, caching caching=caching::none, flag flags=flag::none)
 Construct a handle from a supplied native handle.
 
constexpr io_handle (handle &&o) noexcept
 Explicit conversion from handle permitted.
 
 io_handle (io_handle &&)=default
 Move construction permitted.
 
io_handleoperator= (io_handle &&)=default
 Move assignment permitted.
 
virtual size_t max_buffers () const noexcept
 The maximum number of buffers which a single read or write syscall can process at a time for this specific open handle. On POSIX, this is known as IOV_MAX. More...
 
virtual io_result< buffers_typeread (io_request< buffers_type > reqs, deadline d=deadline()) noexcept
 Read data from the open handle. More...
 
io_result< buffer_typeread (extent_type offset, char *data, size_type bytes, deadline d=deadline()) noexcept
 
virtual io_result< const_buffers_typewrite (io_request< const_buffers_type > reqs, deadline d=deadline()) noexcept
 Write data to the open handle. More...
 
io_result< const_buffer_typewrite (extent_type offset, const char *data, size_type bytes, deadline d=deadline()) noexcept
 
virtual io_result< const_buffers_typebarrier (io_request< const_buffers_type > reqs=io_request< const_buffers_type >(), bool wait_for_device=false, bool and_metadata=false, deadline d=deadline()) noexcept=0
 Issue a write reordering barrier such that writes preceding the barrier will reach storage before writes after this barrier. More...
 
virtual result< extent_guardlock (extent_type offset, extent_type bytes, bool exclusive=true, deadline d=deadline()) noexcept
 Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes through the same semantics as the underlying OS call, including any POSIX insanity present on your platform: More...
 
result< extent_guardtry_lock (extent_type offset, extent_type bytes, bool exclusive=true) noexcept
 
result< extent_guardlock (io_request< buffers_type > reqs, deadline d=deadline()) noexcept
 
result< extent_guardlock (io_request< const_buffers_type > reqs, deadline d=deadline()) noexcept
 
virtual void unlock (extent_type offset, extent_type bytes) noexcept
 Unlocks a byte range previously locked. More...
 
void swap (handle &o) noexcept
 Swap with another instance.
 
virtual result< path_typecurrent_path () const noexcept
 
virtual result< void > close () noexcept
 Immediately close the native handle type managed by this handle.
 
result< handleclone () const noexcept
 
virtual native_handle_type release () noexcept
 Release the native handle type managed by this handle.
 
bool is_valid () const noexcept
 True if the handle is valid (and usually open)
 
bool is_readable () const noexcept
 True if the handle is readable.
 
bool is_writable () const noexcept
 True if the handle is writable.
 
bool is_append_only () const noexcept
 True if the handle is append only.
 
virtual result< void > set_append_only (bool enable) noexcept
 
bool is_overlapped () const noexcept
 True if overlapped.
 
bool is_seekable () const noexcept
 True if seekable.
 
bool requires_aligned_io () const noexcept
 True if requires aligned i/o.
 
bool is_regular () const noexcept
 True if a regular file or device.
 
bool is_directory () const noexcept
 True if a directory.
 
bool is_symlink () const noexcept
 True if a symlink.
 
bool is_multiplexer () const noexcept
 True if a multiplexer like BSD kqueues, Linux epoll or Windows IOCP.
 
bool is_process () const noexcept
 True if a process.
 
bool is_section () const noexcept
 True if a memory section.
 
caching kernel_caching () const noexcept
 Kernel cache strategy used by this handle.
 
bool are_reads_from_cache () const noexcept
 True if the handle uses the kernel page cache for reads.
 
bool are_writes_durable () const noexcept
 True if writes are safely on storage on completion.
 
bool are_safety_fsyncs_issued () const noexcept
 True if issuing safety fsyncs is on.
 
flag flags () const noexcept
 The flags this handle was opened with.
 
native_handle_type native_handle () const noexcept
 The native handle used by this handle.
 

Protected Attributes

caching _caching {caching::none}
 
flag _flags {flag::none}
 
native_handle_type _v
 

Detailed Description

A handle to something capable of scatter-gather i/o.

Constructor & Destructor Documentation

◆ io_handle()

constexpr afio_v2_xxx::io_handle::io_handle ( )
default

Default constructor.

Todo:
Why is io_result<buffers_type> not a standard layout type?

Member Function Documentation

◆ barrier()

virtual io_result<const_buffers_type> afio_v2_xxx::io_handle::barrier ( io_request< const_buffers_type reqs = io_requestconst_buffers_type >(),
bool  wait_for_device = false,
bool  and_metadata = false,
deadline  d = deadline() 
)
pure virtualnoexcept

Issue a write reordering barrier such that writes preceding the barrier will reach storage before writes after this barrier.

Warning
Assume that this call is a no-op. It is not reliably implemented in many common use cases, for example if your code is running inside a LXC container, or if the user has mounted the filing system with non-default options. Instead open the handle with caching::reads which means that all writes form a strict sequential order not completing until acknowledged by the storage device. Filing system can and do use different algorithms to give much better performance with caching::reads, some (e.g. ZFS) spectacularly better.
Let me repeat again: consider this call to be a hint to poke the kernel with a stick to go start to do some work sooner rather than later. It may be ignored entirely.
For portability, you can only assume that barriers write order for a single handle instance. You cannot assume that barriers write order across multiple handles to the same inode, or across processes.
Returns
The buffers barriered, which may not be the buffers input. The size of each scatter-gather buffer is updated with the number of bytes of that buffer barriered.
Parameters
reqsA scatter-gather and offset request for what range to barrier. May be ignored on some platforms which always write barrier the entire file. Supplying a default initialised reqs write barriers the entire file.
wait_for_deviceTrue if you want the call to wait until data reaches storage and that storage has acknowledged the data is physically written. Slow.
and_metadataTrue if you want the call to sync the metadata for retrieving the writes before the barrier after a sudden power loss event. Slow.
dAn optional deadline by which the i/o must complete, else it is cancelled. Note function may return significantly after this deadline if the i/o takes long to cancel.
Errors returnable
Any of the values POSIX fdatasync() or Windows NtFlushBuffersFileEx() can return.
Memory Allocations
None.

◆ clone()

result<handle> afio_v2_xxx::handle::clone ( ) const
inlinenoexceptinherited

Clone this handle (copy constructor is disabled to avoid accidental copying)

Errors returnable
Any of the values POSIX dup() or DuplicateHandle() can return.

◆ current_path()

virtual result<path_type> afio_v2_xxx::handle::current_path ( ) const
inlinevirtualnoexceptinherited

Returns the current path of the open handle as said by the operating system. Note that you are NOT guaranteed that any path refreshed bears any resemblance to the original, some operating systems will return some different path which still reaches the same inode via some other route e.g. hardlinks, dereferenced symbolic links, etc. Windows and Linux correctly track changes to the specific path the handle was opened with, not getting confused by other hard links. MacOS nearly gets it right, but under some circumstances e.g. renaming may switch to a different hard link's path which is almost certainly a bug.

If AFIO was not able to determine the current path for this open handle e.g. the inode has been unlinked, it returns an empty path. Be aware that FreeBSD can return an empty (deleted) path for file inodes no longer cached by the kernel path cache, AFIO cannot detect the difference. FreeBSD will also return any path leading to the inode if it is hard linked. FreeBSD does implement path retrieval for directory inodes correctly however, and see algorithm::stablized_path<T> for a handle adapter which makes use of that.

On Linux if /proc is not mounted, this call fails with an error. All APIs in AFIO which require the use of current_path() can be told to not use it e.g. flag::disable_safety_unlinks. It is up to you to detect if current_path() is not working, and to change how you call AFIO appropriately.

Warning
This call is expensive, it always asks the kernel for the current path, and no checking is done to ensure what the kernel returns is accurate or even sensible. Be aware that despite these precautions, paths are unstable and can change randomly at any moment. Most code written to use absolute file systems paths is racy, so don't do it, use path_handle to fix a base location on the file system and work from that anchor instead!
Memory Allocations
At least one malloc for the path_type, likely several more.
See also
algorithm::cached_parent_handle_adapter<T> which overrides this with an implementation based on retrieving the current path of a cached handle to the parent directory. On platforms with instability or failure to retrieve the correct current path for regular files, the cached parent handle adapter works around the problem by taking advantage of directory inodes not having the same instability problems on any platform.

◆ lock() [1/3]

virtual result<extent_guard> afio_v2_xxx::io_handle::lock ( extent_type  offset,
extent_type  bytes,
bool  exclusive = true,
deadline  d = deadline() 
)
inlinevirtualnoexcept

Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes through the same semantics as the underlying OS call, including any POSIX insanity present on your platform:

  • Any fd closed on an inode must release all byte range locks on that inode for all other fds. If your OS isn't new enough to support the non-insane lock API, flag::byte_lock_insanity will be set in flags() after the first call to this function.
  • Threads replace each other's locks, indeed locks replace each other's locks.

You almost cetainly should use your choice of an algorithm::shared_fs_mutex::* instead of this as those are more portable and performant.

Warning
This is a low-level API which you should not use directly in portable code. Another issue is that atomic lock upgrade/downgrade, if your platform implements that (you should assume it does not in portable code), means that on POSIX you need to release the old extent_guard after creating a new one over the same byte range, otherwise the old extent_guard's destructor will simply unlock the range entirely. On Windows however upgrade/downgrade locks overlay, so on that platform you must not release the old extent_guard. Look into algorithm::shared_fs_mutex::safe_byte_ranges for a portable solution.
Returns
An extent guard, the destruction of which will call unlock().
Parameters
offsetThe offset to lock. Note that on POSIX the top bit is always cleared before use as POSIX uses signed transport for offsets. If you want an advisory rather than mandatory lock on Windows, one technique is to force top bit set so the region you lock is not the one you will i/o - obviously this reduces maximum file size to (2^63)-1.
bytesThe number of bytes to lock. Zero means lock the entire file using any more efficient alternative algorithm where available on your platform (specifically, on BSD and OS X use flock() for non-insane semantics).
exclusiveWhether the lock is to be exclusive.
dAn optional deadline by which the lock must complete, else it is cancelled.
Errors returnable
Any of the values POSIX fcntl() can return, errc::timed_out, errc::not_supported may be returned if deadline i/o is not possible with this particular handle configuration (e.g. non-overlapped HANDLE on Windows).
Memory Allocations
The default synchronous implementation in file_handle performs no memory allocation. The asynchronous implementation in async_file_handle performs one calloc and one free.

◆ lock() [2/3]

result<extent_guard> afio_v2_xxx::io_handle::lock ( io_request< buffers_type reqs,
deadline  d = deadline() 
)
inlinenoexcept

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

408  {
409  size_t bytes = 0;
410  for(auto &i : reqs.buffers)
411  {
412  if(bytes + i.len < bytes)
413  {
414  return std::errc::value_too_large;
415  }
416  bytes += i.len;
417  }
418  return lock(reqs.offset, bytes, false, d);
419  }
virtual result< extent_guard > lock(extent_type offset, extent_type bytes, bool exclusive=true, deadline d=deadline()) noexcept
Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes throu...

◆ lock() [3/3]

result<extent_guard> afio_v2_xxx::io_handle::lock ( io_request< const_buffers_type reqs,
deadline  d = deadline() 
)
inlinenoexcept

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

422  {
423  size_t bytes = 0;
424  for(auto &i : reqs.buffers)
425  {
426  if(bytes + i.len < bytes)
427  {
428  return std::errc::value_too_large;
429  }
430  bytes += i.len;
431  }
432  return lock(reqs.offset, bytes, true, d);
433  }
virtual result< extent_guard > lock(extent_type offset, extent_type bytes, bool exclusive=true, deadline d=deadline()) noexcept
Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes throu...

◆ max_buffers()

virtual size_t afio_v2_xxx::io_handle::max_buffers ( ) const
inlinevirtualnoexcept

The maximum number of buffers which a single read or write syscall can process at a time for this specific open handle. On POSIX, this is known as IOV_MAX.

Note that the actual number of buffers accepted for a read or a write may be significantly lower than this system-defined limit, depending on available resources. The read() or write() call will return the buffers accepted.

Note also that some OSs will error out if you supply more than this limit to read() or write(), but other OSs do not. Some OSs guarantee that each i/o syscall has effects atomically visible or not to other i/o, other OSs do not.

Microsoft Windows does not implement scatter-gather file i/o syscalls except for unbuffered i/o. Thus this function will always return 1 in that situation.

◆ read() [1/2]

virtual io_result<buffers_type> afio_v2_xxx::io_handle::read ( io_request< buffers_type reqs,
deadline  d = deadline() 
)
inlinevirtualnoexcept

Read data from the open handle.

Warning
Depending on the implementation backend, very different buffers may be returned than you supplied. You should always use the buffers returned and assume that they point to different memory and that each buffer's size will have changed.
Returns
The buffers read, which may not be the buffers input. The size of each scatter-gather buffer is updated with the number of bytes of that buffer transferred, and the pointer to the data may be completely different to what was submitted (e.g. it may point into a memory map).
Parameters
reqsA scatter-gather and offset request.
dAn optional deadline by which the i/o must complete, else it is cancelled. Note function may return significantly after this deadline if the i/o takes long to cancel.
Errors returnable
Any of the values POSIX read() can return, errc::timed_out, errc::operation_canceled. errc::not_supported may be returned if deadline i/o is not possible with this particular handle configuration (e.g. reading from regular files on POSIX or reading from a non-overlapped HANDLE on Windows).
Memory Allocations
The default synchronous implementation in file_handle performs no memory allocation. The asynchronous implementation in async_file_handle performs one calloc and one free.

◆ read() [2/2]

io_result<buffer_type> afio_v2_xxx::io_handle::read ( extent_type  offset,
char *  data,
size_type  bytes,
deadline  d = deadline() 
)
inlinenoexcept

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

221  {
222  buffer_type _reqs[1] = {{data, bytes}};
223  io_request<buffers_type> reqs(buffers_type(_reqs), offset);
224  OUTCOME_TRY(v, read(reqs, d));
225  return *v.data();
226  }
span< buffer_type > buffers_type
The scatter buffers type used by this handle. Guaranteed to be TrivialType apart from construction...
Definition: io_handle.hpp:76
virtual io_result< buffers_type > read(io_request< buffers_type > reqs, deadline d=deadline()) noexcept
Read data from the open handle.

◆ set_append_only()

virtual result<void> afio_v2_xxx::handle::set_append_only ( bool  enable)
inlinevirtualnoexceptinherited

Changes whether this handle is append only or not.

Warning
On Windows this is implemented as a bit of a hack to make it fast like on POSIX, so make sure you open the handle for read/write originally. Note unlike on POSIX the append_only disposition will be the only one toggled, seekable and readable will remain turned on.
Errors returnable
Whatever POSIX fcntl() returns. On Windows nothing is changed on the handle.
Memory Allocations
No memory allocation.

◆ try_lock()

result<extent_guard> afio_v2_xxx::io_handle::try_lock ( extent_type  offset,
extent_type  bytes,
bool  exclusive = true 
)
inlinenoexcept

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

405 { return lock(offset, bytes, exclusive, deadline(std::chrono::seconds(0))); }
virtual result< extent_guard > lock(extent_type offset, extent_type bytes, bool exclusive=true, deadline d=deadline()) noexcept
Tries to lock the range of bytes specified for shared or exclusive access. Be aware this passes throu...

◆ unlock()

virtual void afio_v2_xxx::io_handle::unlock ( extent_type  offset,
extent_type  bytes 
)
inlinevirtualnoexcept

Unlocks a byte range previously locked.

Parameters
offsetThe offset to unlock. This should be an offset previously locked.
bytesThe number of bytes to unlock. This should be a byte extent previously locked.
Errors returnable
Any of the values POSIX fcntl() can return.
Memory Allocations
None.

◆ write() [1/2]

virtual io_result<const_buffers_type> afio_v2_xxx::io_handle::write ( io_request< const_buffers_type reqs,
deadline  d = deadline() 
)
inlinevirtualnoexcept

Write data to the open handle.

Warning
Depending on the implementation backend, not all of the buffers input may be written and the some buffers at the end of the returned buffers may return with zero bytes written. For example, with a zeroed deadline, some backends may only consume as many buffers as the system has available write slots for, thus for those backends this call is "non-blocking" in the sense that it will return immediately even if it could not schedule a single buffer write. Another example is that some implementations will not auto-extend the length of a file when a write exceeds the maximum extent, you will need to issue a truncate(newsize) first.
Returns
The buffers written, which may not be the buffers input. The size of each scatter-gather buffer is updated with the number of bytes of that buffer transferred.
Parameters
reqsA scatter-gather and offset request.
dAn optional deadline by which the i/o must complete, else it is cancelled. Note function may return significantly after this deadline if the i/o takes long to cancel.
Errors returnable
Any of the values POSIX write() can return, errc::timed_out, errc::operation_canceled. errc::not_supported may be returned if deadline i/o is not possible with this particular handle configuration (e.g. writing to regular files on POSIX or writing to a non-overlapped HANDLE on Windows).
Memory Allocations
The default synchronous implementation in file_handle performs no memory allocation. The asynchronous implementation in async_file_handle performs one calloc and one free.

◆ write() [2/2]

io_result<const_buffer_type> afio_v2_xxx::io_handle::write ( extent_type  offset,
const char *  data,
size_type  bytes,
deadline  d = deadline() 
)
inlinenoexcept

This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.

253  {
254  const_buffer_type _reqs[1] = {{data, bytes}};
255  io_request<const_buffers_type> reqs(const_buffers_type(_reqs), offset);
256  OUTCOME_TRY(v, write(reqs, d));
257  return *v.data();
258  }
span< const_buffer_type > const_buffers_type
The gather buffers type used by this handle. Guaranteed to be TrivialType apart from construction...
Definition: io_handle.hpp:78
virtual io_result< const_buffers_type > write(io_request< const_buffers_type > reqs, deadline d=deadline()) noexcept
Write data to the open handle.

The documentation for this class was generated from the following file: