README
¶
The renameio Go package provides a way to atomically create or replace a file or
symbolic link.
Atomicity vs durability
renameio concerns itself only with atomicity, i.e. making sure applications
never see unexpected file content (a half-written file, or a 0-byte file).
As a practical example, consider https://manpages.debian.org/: if there is a power outage while the site is updating, we are okay with losing the manpages which were being rendered at the time of the power outage. They will be added in a later run of the software. We are not okay with having a manpage replaced by a 0-byte file under any circumstances, though.
Advantages of this package
There are other packages for atomically replacing files, and sometimes ad-hoc implementations can be found in programs.
A naive approach to the problem is to create a temporary file followed by a call
to os.Rename(). However, there are a number of subtleties which make the
correct sequence of operations hard to identify:
-
The temporary file should be removed when an error occurs, but a remove must not be attempted if the rename succeeded, as a new file might have been created with the same name. This renders a throwaway
defer os.Remove(t.Name())insufficient; state must be kept. -
The temporary file must be created on the same file system (same mount point) for the rename to work, but the TMPDIR environment variable should still be respected, e.g. to direct temporary files into a separate directory outside of the webserver’s document root but on the same file system.
-
On POSIX operating systems, the
fsyncsystem call must be used to ensure that theos.Rename()call will not result in a 0-length file.
This package attempts to get all of these details right, provides an intuitive, yet flexible API and caters to use-cases where high performance is required.
Major changes in v2
With major version renameio/v2, renameio.WriteFile changes the way that
permissions are handled. Before version 2, files were created with the
permissions passed to the function, ignoring the
umask. From version 2 onwards, these
permissions are further modified by process' umask (usually the user's
preferred umask).
If you were relying on the umask being ignored, add the
renameio.IgnoreUmask() option to your renameio.WriteFile calls when
upgrading to v2.
Windows support
It is not possible to reliably write files atomically on
Windows, and
chmod is not reliably supported by the Go standard library on
Windows.
As it is not possible to provide a correct implementation, this package does not export any functions on Windows.
Disclaimer
This is not an official Google product (experimental or otherwise), it is just code that happens to be owned by Google.
This project is not affiliated with the Go project.
Documentation
¶
Overview ¶
Package renameio provides a way to atomically create or replace a file or symbolic link.
Caveat: this package requires the file system rename(2) implementation to be atomic. Notably, this is not the case when using NFS with multiple clients: https://stackoverflow.com/a/41396801
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Symlink ¶
Symlink wraps os.Symlink, replacing an existing symlink with the same name atomically (os.Symlink fails when newname already exists, at least on Linux).
func TempDir ¶
TempDir checks whether os.TempDir() can be used as a temporary directory for later atomically replacing files within dest. If no (os.TempDir() resides on a different mount point), dest is returned.
Note that the returned value ceases to be valid once either os.TempDir() changes (e.g. on Linux, once the TMPDIR environment variable changes) or the file system is unmounted.
Types ¶
type Option ¶
type Option interface {
// contains filtered or unexported methods
}
Option is the interface implemented by all configuration function return values.
func IgnoreUmask ¶
func IgnoreUmask() Option
IgnoreUmask causes the permissions configured using WithPermissions to be applied directly without applying the umask.
func WithExistingPermissions ¶
func WithExistingPermissions() Option
WithExistingPermissions configures the file creation to try to use the permissions from an already existing target file. If the target file doesn't exist yet or is not a regular file the default permissions are used unless overridden using WithPermissions or WithStaticPermissions.
func WithPermissions ¶
WithPermissions sets the permissions for the target file while respecting the umask(2). Bits set in the umask are removed from the permissions given unless IgnoreUmask is used.
func WithStaticPermissions ¶
WithStaticPermissions sets the permissions for the target file ignoring the umask(2). This is equivalent to calling Chmod() on the file handle or using WithPermissions in combination with IgnoreUmask.
func WithTempDir ¶
WithTempDir configures the directory to use for temporary, uncommitted files. Suitable for using a cached directory from TempDir(filepath.Base(path)).
type PendingFile ¶
PendingFile is a pending temporary file, waiting to replace the destination path in a call to CloseAtomicallyReplace.
func NewPendingFile ¶
func NewPendingFile(path string, opts ...Option) (*PendingFile, error)
NewPendingFile creates a temporary file destined to atomically creating or replacing the destination file at path.
TempDir(filepath.Base(path)) is used to store the temporary file. If you are going to write a large number of files to the same file system, use the result of TempDir(filepath.Base(path)) with the WithTempDir option.
The file's permissions will be (0600 & ^umask). Use WithPermissions, IgnoreUmask, WithStaticPermissions and WithExistingPermissions to control them.
func TempFile ¶
func TempFile(dir, path string) (*PendingFile, error)
TempFile creates a temporary file destined to atomically creating or replacing the destination file at path.
If dir is the empty string, TempDir(filepath.Base(path)) is used. If you are going to write a large number of files to the same file system, store the result of TempDir(filepath.Base(path)) and pass it instead of the empty string.
The file's permissions will be 0600. You can change these by explicitly calling Chmod on the returned PendingFile.
Example (Justone) ¶
persist := func(temperature float64) error {
t, err := renameio.TempFile("", "/srv/www/metrics.txt")
if err != nil {
return err
}
defer t.Cleanup()
if _, err := fmt.Fprintf(t, "temperature_degc %f\n", temperature); err != nil {
return err
}
return t.CloseAtomicallyReplace()
}
// Thanks to the write package, a webserver exposing /srv/www never
// serves an incomplete or missing file.
if err := persist(31.2); err != nil {
log.Fatal(err)
}
Output:
Example (Many) ¶
// Prepare for writing files to /srv/www, effectively caching calls to
// TempDir which TempFile would otherwise need to make.
dir := renameio.TempDir("/srv/www")
persist := func(temperature float64) error {
t, err := renameio.TempFile(dir, "/srv/www/metrics.txt")
if err != nil {
return err
}
defer t.Cleanup()
if _, err := fmt.Fprintf(t, "temperature_degc %f\n", temperature); err != nil {
return err
}
return t.CloseAtomicallyReplace()
}
// Imagine this was an endless loop, reading temperature sensor values.
// Thanks to the write package, a webserver exposing /srv/www never
// serves an incomplete or missing file.
for {
if err := persist(31.2); err != nil {
log.Fatal(err)
}
}
Output:
func (*PendingFile) Cleanup ¶
func (t *PendingFile) Cleanup() error
Cleanup is a no-op if CloseAtomicallyReplace succeeded, and otherwise closes and removes the temporary file.
This method is not safe for concurrent use by multiple goroutines.
func (*PendingFile) CloseAtomicallyReplace ¶
func (t *PendingFile) CloseAtomicallyReplace() error
CloseAtomicallyReplace closes the temporary file and atomically replaces the destination file with it, i.e., a concurrent open(2) call will either open the file previously located at the destination path (if any), or the just written file, but the file will always be present.
This method is not safe for concurrent use by multiple goroutines.
Source Files
Directories