AFIO  v2.00 late alpha
afio_v2_xxx::utils Namespace Reference

Utility routines often useful when using AFIO. More...

Classes

class  page_allocator
 An STL allocator which allocates large TLB page memory.If the operating system is configured to allow it, this type of memory is particularly efficient for doing large scale file i/o. This is because the kernel must normally convert the scatter gather buffers you pass into extended scatter gather buffers as the memory you see as contiguous may not, and probably isn't, actually be contiguous in physical memory. Regions returned by this allocator may be allocated contiguously in physical memory and therefore the kernel can pass through your scatter gather buffers unmodified. More...
 
class  page_allocator< void >
 

Functions

size_t page_size () noexcept
 Returns the smallest page size of this architecture which is useful for calculating direct i/o multiples. More...
 
template<class T >
round_down_to_page_size (T i) noexcept
 Round a value to its next lowest page size multiple.
 
template<class T >
round_up_to_page_size (T i) noexcept
 Round a value to its next highest page size multiple.
 
template<class T >
round_to_page_size (T i) noexcept
 Round a pair of a pointer and a size_t to their nearest page size multiples. The pointer will be rounded down, the size_t upwards.
 
std::vector< size_t > page_sizes (bool only_actually_available=true)
 Returns the page sizes of this architecture which is useful for calculating direct i/o multiples. More...
 
size_t file_buffer_default_size ()
 Returns a reasonable default size for page_allocator, typically the closest page size from page_sizes() to 1Mb. More...
 
void random_fill (char *buffer, size_t bytes) noexcept
 Fills the buffer supplied with cryptographically strong randomness. Uses the OS kernel API. More...
 
std::string random_string (size_t randomlen)
 Returns a cryptographically random string capable of being used as a filename. Essentially random_fill() + to_hex_string(). More...
 
result< void > flush_modified_data () noexcept
 Tries to flush all modified data to the physical device.
 
result< void > drop_filesystem_cache () noexcept
 Tries to flush all modified data to the physical device, and then drop the OS filesystem cache, thus making all future reads come from the physical device. Currently only implemented for Microsoft Windows and Linux. More...
 
template<class T , class U >
bool operator== (const page_allocator< T > &, const page_allocator< U > &) noexcept
 

Detailed Description

Utility routines often useful when using AFIO.

Function Documentation

◆ drop_filesystem_cache()

result<void> afio_v2_xxx::utils::drop_filesystem_cache ( )
inlinenoexcept

Tries to flush all modified data to the physical device, and then drop the OS filesystem cache, thus making all future reads come from the physical device. Currently only implemented for Microsoft Windows and Linux.

Note that the OS specific magic called by this routine generally requires elevated privileges for the calling process. For obvious reasons, calling this will have a severe negative impact on performance, but it's very useful for benchmarking cold cache vs warm cache performance.

◆ file_buffer_default_size()

size_t afio_v2_xxx::utils::file_buffer_default_size ( )
inline

Returns a reasonable default size for page_allocator, typically the closest page size from page_sizes() to 1Mb.

Returns
A value of a TLB large page size close to 1Mb.
Complexity
Whatever the system API takes (one would hope constant time).
Errors returnable
Any error from the operating system or std::bad_alloc.
95  {
96  static size_t size;
97  if(size == 0u)
98  {
99  std::vector<size_t> sizes(page_sizes(true));
100  for(auto &i : sizes)
101  {
102  if(i >= 1024 * 1024)
103  {
104  size = i;
105  break;
106  }
107  }
108  if(size == 0u)
109  {
110  size = 1024 * 1024;
111  }
112  }
113  return size;
114  }
std::vector< size_t > page_sizes(bool only_actually_available=true)
Returns the page sizes of this architecture which is useful for calculating direct i/o multiples...

◆ page_size()

size_t afio_v2_xxx::utils::page_size ( )
inlinenoexcept

Returns the smallest page size of this architecture which is useful for calculating direct i/o multiples.

Returns
The page size of this architecture.
Complexity
Whatever the system API takes (one would hope constant time).

◆ page_sizes()

std::vector<size_t> afio_v2_xxx::utils::page_sizes ( bool  only_actually_available = true)
inline

Returns the page sizes of this architecture which is useful for calculating direct i/o multiples.

Parameters
only_actually_availableOnly return page sizes actually available to the user running this process
Returns
The page sizes of this architecture.
Complexity
Whatever the system API takes (one would hope constant time).
Errors returnable
Any error from the operating system or std::bad_alloc.

◆ random_fill()

void afio_v2_xxx::utils::random_fill ( char *  buffer,
size_t  bytes 
)
inlinenoexcept

Fills the buffer supplied with cryptographically strong randomness. Uses the OS kernel API.

Parameters
bufferA buffer to fill
bytesHow many bytes to fill
Complexity
Whatever the system API takes.
Errors returnable
Any error from the operating system.

◆ random_string()

std::string afio_v2_xxx::utils::random_string ( size_t  randomlen)
inline

Returns a cryptographically random string capable of being used as a filename. Essentially random_fill() + to_hex_string().

Parameters
randomlenThe number of bytes of randomness to use for the string.
Returns
A string representing the randomness at a 2x ratio, so if 32 bytes were requested, this string would be 64 bytes long.
Complexity
Whatever the system API takes.
Errors returnable
Any error from the operating system.
135  {
136  size_t outlen = randomlen * 2;
137  std::string ret(outlen, 0);
138  random_fill(const_cast<char *>(ret.data()), randomlen);
139  QUICKCPPLIB_NAMESPACE::algorithm::string::to_hex_string(const_cast<char *>(ret.data()), outlen, ret.data(), randomlen);
140  return ret;
141  }
void random_fill(char *buffer, size_t bytes) noexcept
Fills the buffer supplied with cryptographically strong randomness. Uses the OS kernel API...