Memory-Mapped-File

Maps a file into virtual memory.  This allows accessing the content of a file as simple as an array.

Includes

You need to include “C-kern/api/io/accessmode.h” (IO-Accessmode) before this file.

Summary
Memory-Mapped-FileMaps a file into virtual memory.
CopyrightThis program is free software.
Files
C-kern/api/io/filesystem/mmfile.hHeader file of Memory-Mapped-File.
C-kern/platform/Linux/io/mmfile.cLinux specific implementation Memory-Mapped-File Linux.
Types
struct mmfile_tExports mmfile_t.
Functions
test
unittest_io_mmfileTest mapping of file into memory.
mmfile_tDescribes a memory mapped file.
addrThe start address of the mapped memory.
sizeSize of the mapped memory.
lifetime
mmfile_FREEStatic initializer for mmfile_t.
init_mmfileOpens a new file and maps it to memory.
initfromio_mmfileMaps a file referenced by sys_iochannel_t into memory.
initsplit_mmfileSplit a memory mapping into two.
initmove_mmfileMoves content of sourcemfile to destmfile.
free_mmfileFrees all mapped memory and closes the file.
query
isfree_mmfileReturns true if mfile == mmfile_FREE.
addr_mmfileReturns the lowest address of the mapped memory.
size_mmfileReturns the size of the mapped memory.
alignedsize_mmfileReturns the size of the mapped memory.
change
seek_mmfileUse it to change offset of the underlying file mapped into memory.
generic
genericcast_mmfileCasts a generic object pointer into pointer to memblock_t.
inline implementation
Macros
addr_mmfileImplements mmfile_t.addr_mmfile.
alignedsize_mmfileImplements mmfile_t.alignedsize_mmfile.
genericcast_mmfileImplements mmfile_t.genericcast_mmfile.
initmove_mmfileImplements mmfile_t.initmove_mmfile.
isfree_mmfileImplements mmfile_t.isfree_mmfile.
size_mmfileImplements mmfile_t.size_mmfile.

Copyright

This program is free software.  You can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.

Author

© 2011 Jörg Seebohn

Files

C-kern/api/io/filesystem/mmfile.h

Header file of Memory-Mapped-File.

C-kern/platform/Linux/io/mmfile.c

Linux specific implementation Memory-Mapped-File Linux.

Types

struct mmfile_t

typedef struct mmfile_t mmfile_t

Exports mmfile_t.

Functions

Summary
test
unittest_io_mmfileTest mapping of file into memory.

test

unittest_io_mmfile

int unittest_io_mmfile(void)

Test mapping of file into memory.

mmfile_t

struct mmfile_t

Describes a memory mapped file.  Memory mapped files must always be readable cause the memory must be initialized before it can be accessed.  Even if you only want to write to it.

If you want to open an executable file use always accessmode <accessmode_RDEX_SHARED>.

TODO: Recovery

In case a read error occurs a SIGBUS is thrown under Linux.  Register special recovery handler for mmfiles => abort + read error !

TODO: memory mapping fails

Add a check function to mmfile_t which checks if memory mapping is possible ! If not then use some »read into buffer fallback operation« in some higher component !!

Summary
addrThe start address of the mapped memory.
sizeSize of the mapped memory.
lifetime
mmfile_FREEStatic initializer for mmfile_t.
init_mmfileOpens a new file and maps it to memory.
initfromio_mmfileMaps a file referenced by sys_iochannel_t into memory.
initsplit_mmfileSplit a memory mapping into two.
initmove_mmfileMoves content of sourcemfile to destmfile.
free_mmfileFrees all mapped memory and closes the file.
query
isfree_mmfileReturns true if mfile == mmfile_FREE.
addr_mmfileReturns the lowest address of the mapped memory.
size_mmfileReturns the size of the mapped memory.
alignedsize_mmfileReturns the size of the mapped memory.
change
seek_mmfileUse it to change offset of the underlying file mapped into memory.
generic
genericcast_mmfileCasts a generic object pointer into pointer to memblock_t.

addr

uint8_t * addr

The start address of the mapped memory.  It is a multiple of pagesize_vm.

size

size_t size

Size of the mapped memory.  size will be a multiple of pagesize_vm except if filesize - file_offset would be < size.  In this case size is truncated to filesize - file_offset (see init_mmfile).

lifetime

mmfile_FREE

#define mmfile_FREE { 0, 0 }

Static initializer for mmfile_t.  Makes calling of free_mmfile a no op.

init_mmfile

int init_mmfile(
   /*out*/mmfile_t *mfile,  
   const char *file_path,  
   off_t file_offset,  
   size_t size,  
   accessmode_e mode,  
   const struct directory_t * relative_to /*0 = >current_working_directory*/
)

Opens a new file and maps it to memory.  Parameter file_path is considered relative to relative_to instead of the current working directory if relative_to is set to a value != 0 and file_path is a relative path

Files can be mapped into memory only page by page.  If the file size is not a multiple of pagesize_vm all memory of the last mapped page which is beyond the size of the underlying file is filled with 0, and writes to that region are not written to the file.

Parameter

mfileMemory mapped file object
file_pathPath of file to be read or written.  A relatice path is relative to relative_to or the current working directory.
file_offsetThe offset of the first byte in the file whcih should be mapped.  Must be a multiple of pagesize_vm.  If file_offset >= filesize then ENODATA is returned => Files with length 0 always generate ENODATA error.
sizeThe number of bytes which should be mapped.  The mapping is always done in chunks of pagesize_vm. size (in case it is != 0) is increased until it is a multiple of pagesize_vm.  If file_offset + size > filesize then size is silently truncated (file_soffset + size == filesize).  If size is set to 0 then this means to map the whole file starting from file_offset.  If the file is bigger than could be mapped into memory ENOMEM is returned.
relative_toIf this parameter is != 0 and file_path is a relative path then file_path is considered relative to the path determined by this parameter.
modeDetermines if the file is opened for reading or (reading and writing).  Allowed values are accessmode_READ, accessmode_RDWR_PRIVATE and accessmode_RDWR_SHARED.

initfromio_mmfile

int initfromio_mmfile(/*out*/mmfile_t *mfile,
sys_iochannel_t fd,
off_t file_offset,
size_t size,
accessmode_e mode)

Maps a file referenced by sys_iochannel_t into memory.  The function does the same as init_mmfile except it does not open a file but takes a file descriptor to the already opened file.  The file must always be opened with read access and also write access in case mode contains accessmode_WRITE.

Attention

If the file_offset + size is bigger than the size of the underlying file accessing the memory which has no file backing is undefinded.  The operating system creates a bus error exception in cases where a whole memory page has no backing file object.  If the size is set to 0 no mapping is done at all and mmfile_t is initialized with a 0 (<memblock_FREE>).

initsplit_mmfile

int initsplit_mmfile(/*out*/mmfile_t *destheadmfile,
/*out*/mmfile_t *desttailmfile,
size_t headsize,
mmfile_t *sourcemfile)

Split a memory mapping into two.  After return destheadmfile maps the first headsize bytes. headsize must be a multiple of pagesize_vm. desttailmfile maps the last (size_mmfile(sourcemfile)-headsize) bytes. sourcemfile is set to mmfile_FREE if it is not equal to destheadmfile or desttailmfile.

initmove_mmfile

void initmove_mmfile(/*out*/mmfile_t * restrict destmfile,
mmfile_t * restrict sourcemfile)

Moves content of sourcemfile to destmfile. sourcemfile is also reset to mmfile_FREE.

free_mmfile

int free_mmfile(mmfile_t *mfile)

Frees all mapped memory and closes the file.

query

isfree_mmfile

bool isfree_mmfile(const mmfile_t *mfile)

Returns true if mfile == mmfile_FREE.

addr_mmfile

uint8_t * addr_mmfile(const mmfile_t *mfile)

Returns the lowest address of the mapped memory.  The memory is always mapped in chunks of pagesize_vm.  The memory which can be accessed is at least [addr_mmfile ..  <addr_mmfile>+<size_mmfile>].

size_mmfile

size_t size_mmfile(const mmfile_t *mfile)

Returns the size of the mapped memory.  The memory which corresponds to the underlying file is exactly [addr_mmfile ..  <addr_mmfile>+<size_mmfile>].

alignedsize_mmfile

size_t alignedsize_mmfile(const mmfile_t *mfile)

Returns the size of the mapped memory.  This value is a multiple of pagesize_vm and it is >= size_mmfile.  The mapped memory region is [addr_mmfile ..  <addr_mmfile>+<alignedsize_mmfile>] but only [addr_mmfile ..  <addr_mmfile>+<size_mmfile>] corresponds to the underlying file.

change

seek_mmfile

int seek_mmfile(mmfile_t *mfile,
sys_iochannel_t fd,
off_t file_offset,
accessmode_e mode)

Use it to change offset of the underlying file mapped into memory.  The file_offset must be a multiple of pagesize_vm -- see also initfromio_mmfile.

Attention

If size_file(fd)file_offset (see file_t.size_file) is smaller than the value returned by size_mmfile accessing outside the memory address range can result in a SIGBUS (see initfromio_mmfile).  The solution is to handle the size on a higher level after a change of the file offset.

generic

genericcast_mmfile

mmfile_t * genericcast_mmfile(void *obj,
IDNAME nameprefix,
TYPEQUALIFIER qualifier)

Casts a generic object pointer into pointer to memblock_t.  Set qual to const if you want to cast a const pointer.  The object must have two data members with access path “obj->nameprefix##addr” and “obj->nameprefix##size” of the same type as mmfile_t and in the same order.

Macros

addr_mmfile

#define addr_mmfile(mfile) ((mfile)->addr)

Implements mmfile_t.addr_mmfile.

alignedsize_mmfile

#define alignedsize_mmfile(
   mfile
) ((size_mmfile(mfile) + (pagesize_vm()-1)) & ~(pagesize_vm()-1))

Implements mmfile_t.alignedsize_mmfile.

genericcast_mmfile

#define genericcast_mmfile(
   obj,
   nameprefix,
   qualifier
) ( __extension__ ({ typeof(obj) _obj = (obj) ; static_assert( sizeof(_obj->nameprefix##addr) == sizeof(((mmfile_t*)0)->addr) && 0 == offsetof(mmfile_t, addr), && (typeof(_obj->nameprefix##addr)*)0 == (uint8_t**)0 && sizeof(_obj->nameprefix##size) == sizeof(((mmfile_t*)0)->size) && offsetof(mmfile_t, size) == ((uintptr_t)&_obj->nameprefix##size) -((uintptr_t)&_obj->nameprefix##addr) && (typeof(_obj->nameprefix##size)*)0 == (size_t**)0 "structure is compatible") ; (qualifier mmfile_t *)(&_obj->nameprefix##addr) ; }))

Implements mmfile_t.genericcast_mmfile.

initmove_mmfile

#define initmove_mmfile(
   destmfile,
   sourcemfile
) do { mmfile_t * _sourcemfile = (sourcemfile); *(destmfile) = *(_sourcemfile); *(_sourcemfile) = (mmfile_t) mmfile_FREE; } while (0)

Implements mmfile_t.initmove_mmfile.

isfree_mmfile

#define isfree_mmfile(
   mfile
) (0 == addr_mmfile(mfile) && 0 == size_mmfile(mfile))

Implements mmfile_t.isfree_mmfile.

size_mmfile

#define size_mmfile(mfile) ((mfile)->size)

Implements mmfile_t.size_mmfile.

Maps a file into virtual memory.
Implements Memory-Mapped-File on Linux.
typedef struct mmfile_t mmfile_t
Exports mmfile_t.
struct mmfile_t
Describes a memory mapped file.
int unittest_io_mmfile(void)
Test mapping of file into memory.
uint8_t * addr
The start address of the mapped memory.
size_t size
Size of the mapped memory.
#define mmfile_FREE { 0, 0 }
Static initializer for mmfile_t.
int init_mmfile(
   /*out*/mmfile_t *mfile,  
   const char *file_path,  
   off_t file_offset,  
   size_t size,  
   accessmode_e mode,  
   const struct directory_t * relative_to /*0 = >current_working_directory*/
)
Opens a new file and maps it to memory.
int initfromio_mmfile(/*out*/mmfile_t *mfile,
sys_iochannel_t fd,
off_t file_offset,
size_t size,
accessmode_e mode)
Maps a file referenced by sys_iochannel_t into memory.
#define sys_iochannel_t int
Choose Posix file descriptor type.
int initsplit_mmfile(/*out*/mmfile_t *destheadmfile,
/*out*/mmfile_t *desttailmfile,
size_t headsize,
mmfile_t *sourcemfile)
Split a memory mapping into two.
void initmove_mmfile(/*out*/mmfile_t * restrict destmfile,
mmfile_t * restrict sourcemfile)
Moves content of sourcemfile to destmfile.
int free_mmfile(mmfile_t *mfile)
Frees all mapped memory and closes the file.
bool isfree_mmfile(const mmfile_t *mfile)
Returns true if mfile == mmfile_FREE.
uint8_t * addr_mmfile(const mmfile_t *mfile)
Returns the lowest address of the mapped memory.
size_t size_mmfile(const mmfile_t *mfile)
Returns the size of the mapped memory.
size_t alignedsize_mmfile(const mmfile_t *mfile)
Returns the size of the mapped memory.
int seek_mmfile(mmfile_t *mfile,
sys_iochannel_t fd,
off_t file_offset,
accessmode_e mode)
Use it to change offset of the underlying file mapped into memory.
mmfile_t * genericcast_mmfile(void *obj,
IDNAME nameprefix,
TYPEQUALIFIER qualifier)
Casts a generic object pointer into pointer to memblock_t.
struct memblock_t
Describes memory block.
#define addr_mmfile(mfile) ((mfile)->addr)
Implements mmfile_t.addr_mmfile.
#define alignedsize_mmfile(
   mfile
) ((size_mmfile(mfile) + (pagesize_vm()-1)) & ~(pagesize_vm()-1))
Implements mmfile_t.alignedsize_mmfile.
#define genericcast_mmfile(
   obj,
   nameprefix,
   qualifier
) ( __extension__ ({ typeof(obj) _obj = (obj) ; static_assert( sizeof(_obj->nameprefix##addr) == sizeof(((mmfile_t*)0)->addr) && 0 == offsetof(mmfile_t, addr), && (typeof(_obj->nameprefix##addr)*)0 == (uint8_t**)0 && sizeof(_obj->nameprefix##size) == sizeof(((mmfile_t*)0)->size) && offsetof(mmfile_t, size) == ((uintptr_t)&_obj->nameprefix##size) -((uintptr_t)&_obj->nameprefix##addr) && (typeof(_obj->nameprefix##size)*)0 == (size_t**)0 "structure is compatible") ; (qualifier mmfile_t *)(&_obj->nameprefix##addr) ; }))
Implements mmfile_t.genericcast_mmfile.
#define initmove_mmfile(
   destmfile,
   sourcemfile
) do { mmfile_t * _sourcemfile = (sourcemfile); *(destmfile) = *(_sourcemfile); *(_sourcemfile) = (mmfile_t) mmfile_FREE; } while (0)
Implements mmfile_t.initmove_mmfile.
#define isfree_mmfile(
   mfile
) (0 == addr_mmfile(mfile) && 0 == size_mmfile(mfile))
Implements mmfile_t.isfree_mmfile.
#define size_mmfile(mfile) ((mfile)->size)
Implements mmfile_t.size_mmfile.
Defines how you can access a data block.
uint32_t pagesize_vm(void)
Returns the virtual memory page size supported by the underlying system.
Allows for reading (only).
Combination of accessmode_RDWR and accessmode_PRIVATE.
Combination of accessmode_RDWR and accessmode_SHARED.
Allows for writing (only).
int size_file(const file_t fileobj,
/*out*/off_t *file_size)
Returns the size in bytes of the file.
Close