Files

The files module handles filesystem state, file uploads and template generation.

files.directory

Add/remove/update directories.

files.directory(
    name, present=True, assume_present=False, user=None, group=None, mode=None,
    recursive=False
)
  • name: name/path of the remote folder

  • present: whether the folder should exist

  • assume_present: whether to assume the directory exists

  • user: user to own the folder

  • group: group to own the folder

  • mode: permissions of the folder

  • recursive: recursively apply user/group/mode

Examples:

files.directory(
    {'Ensure the /tmp/dir_that_we_want_removed is removed'},
    '/tmp/dir_that_we_want_removed',
    present=False,
)

files.directory(
    {'Ensure /web exists'},
    '/web',
    user='myweb',
    group='myweb',
)

# multiple directories
dirs = ['/netboot/tftp', '/netboot/nfs']
for dir in dirs:
    files.directory(
        {'Ensure the directory `{}` exists'.format(dir)},
        dir,
    )

files.download

Download files from remote locations using curl or wget.

files.download(
    source_url, destination, user=None, group=None, mode=None, cache_time=None, force=False,
    sha256sum=None, sha1sum=None, md5sum=None
)
  • source_url: source URL of the file

  • destination: where to save the file

  • user: user to own the files

  • group: group to own the files

  • mode: permissions of the files

  • cache_time: if the file exists already, re-download after this time (in seconds)

  • force: always download the file, even if it already exists

  • **sha**256sum: sha256 hash to checksum the downloaded file against

  • **sha**1sum: sha1 hash to checksum the downloaded file against

  • **md**5sum: md5 hash to checksum the downloaded file against

Example:

files.download(
    {'Download the Docker repo file'},
    'https://download.docker.com/linux/centos/docker-ce.repo',
    '/etc/yum.repos.d/docker-ce.repo',
)

files.file

Add/remove/update files.

files.file(
    name, present=True, assume_present=False, user=None, group=None, mode=None, touch=False,
    create_remote_dir=False
)
  • name: name/path of the remote file

  • present: whether the file should exist

  • assume_present: whether to assume the file exists

  • user: user to own the files

  • group: group to own the files

  • mode: permissions of the files as an integer, eg: 755

  • touch: whether to touch the file

  • create_remote_dir: create the remote directory if it doesn’t exist

create_remote_dir:

If the remote directory does not exist it will be created using the same user & group as passed to files.put. The mode will not be copied over, if this is required call files.directory separately.

Example:

# Note: The directory /tmp/secret will get created with the default umask.
files.file(
    {'Create /tmp/secret/file'},
    '/tmp/secret/file',
    mode='600',
    user='root',
    group='root',
    touch=True,
    create_remote_dir=True,
)

files.get

Download a file from the remote system.

files.get(remote_filename, local_filename, add_deploy_dir=True, create_local_dir=False, force=False)
  • remote_filename: the remote filename to download

  • local_filename: the local filename to download the file to

  • add_deploy_dir: local_filename is relative to the deploy directory

  • create_local_dir: create the local directory if it doesn’t exist

  • force: always download the file, even if the local copy matches

Note:

This operation is not suitable for large files as it may involve copying the remote file before downloading it.

Example:

files.get(
    {'Download a file from a remote'},
    '/etc/centos-release',
    '/tmp/whocares',
)

files.line

Ensure lines in files using grep to locate and sed to replace.

files.line(name, line, present=True, replace=None, flags=None)
  • name: target remote file to edit

  • line: string or regex matching the target line

  • present: whether the line should be in the file

  • replace: text to replace entire matching lines when present=True

  • flags: list of flags to pass to sed when replacing/deleting

Regex line matching:

Unless line matches a line (starts with ^, ends $), pyinfra will wrap it such that it does, like: ^.*LINE.*$. This means we don’t swap parts of lines out. To change bits of lines, see files.replace.

Regex line escaping:

If matching special characters (eg a crontab line containing *), remember to escape it first using Python’s re.escape.

Examples:

# prepare to do some maintenance
maintenance_line = 'SYSTEM IS DOWN FOR MAINTENANCE'
files.line(
    {'Add the down-for-maintence line in /etc/motd'},
    '/etc/motd',
    maintenance_line,
)

# Then, after the mantenance is done, remove the maintenance line
files.line(
    {'Remove the down-for-maintenance line in /etc/motd'},
    '/etc/motd',
    maintenance_line,
    replace='',
    present=False,
)

# example where there is '*' in the line
files.line(
    {'Ensure /netboot/nfs is in /etc/exports'},
    '/etc/exports',
    r'/netboot/nfs .*',
    replace='/netboot/nfs *(ro,sync,no_wdelay,insecure_locks,no_root_squash,'
    'insecure,no_subtree_check)',
)

files.line(
    {'Ensure myweb can run /usr/bin/python3 without password'},
    '/etc/sudoers',
    r'myweb .*',
    replace='myweb ALL=(ALL) NOPASSWD: /usr/bin/python3',
)

# example when there are double quotes (")
line = 'QUOTAUSER=""'
results = files.line(
    {'Example with double quotes (")'},
    '/etc/adduser.conf',
    '^{}$'.format(line),
    replace=line,
)
print(results.changed)

files.put

Upload a local file to the remote system.

files.put(
    local_filename, remote_filename, user=None, group=None, mode=None, add_deploy_dir=True,
    create_remote_dir=False, force=False, assume_exists=False
)
  • local_filename: local filename

  • remote_filename: remote filename

  • user: user to own the files

  • group: group to own the files

  • mode: permissions of the files

  • add_deploy_dir: local_filename is relative to the deploy directory

  • create_remote_dir: create the remote directory if it doesn’t exist

  • force: always upload the file, even if the remote copy matches

  • assume_exists: whether to assume the local file exists

create_remote_dir:

If the remote directory does not exist it will be created using the same user & group as passed to files.put. The mode will not be copied over, if this is required call files.directory separately.

Note:

This operation is not suitable for large files as it may involve copying the file before uploading it.

Examples:

# Note: This requires a 'files/motd' file on the local filesystem
files.put(
    {'Update the message of the day file'},
    'files/motd',
    '/etc/motd',
    mode='644',
)

files.replace

A simple shortcut for replacing text in files with sed.

files.replace(name, match, replace, flags=None)
  • name: target remote file to edit

  • match: text/regex to match for

  • replace: text to replace with

  • flags: list of flags to pass to sed

Example:

files.replace(
    {'Change part of a line in a file'},
    '/etc/motd',
    'verboten',
    'forbidden',
)

files.sync

Syncs a local directory with a remote one, with delete support. Note that delete will remove extra files on the remote side, but not extra directories.

files.sync(
    source, destination, user=None, group=None, mode=None, delete=False, exclude=None,
    exclude_dir=None, add_deploy_dir=True
)
  • source: local directory to sync

  • destination: remote directory to sync to

  • user: user to own the files and directories

  • group: group to own the files and directories

  • mode: permissions of the files

  • delete: delete remote files not present locally

  • exclude: string or list/tuple of strings to match & exclude files (eg *.pyc)

  • exclude_dir: string or list/tuple of strings to match & exclude directories (eg node_modules)

Example:

# Sync local files/tempdir to remote /tmp/tempdir
files.sync(
    {'Sync a local directory with remote'},
    'files/tempdir',
    '/tmp/tempdir',
)

files.template

Generate a template using jinja2 and write it to the remote system.

files.template(
    template_filename, remote_filename, user=None, group=None, mode=None,
    create_remote_dir=False
)
  • template_filename: local template filename

  • remote_filename: remote filename

  • user: user to own the files

  • group: group to own the files

  • mode: permissions of the files

  • create_remote_dir: create the remote directory if it doesn’t exist

create_remote_dir:

If the remote directory does not exist it will be created using the same user & group as passed to files.put. The mode will not be copied over, if this is required call files.directory separately.

Notes:

Common convention is to store templates in a “templates” directory and have a filename suffix with ‘.j2’ (for jinja2).

For information on the template syntax, see the jinja2 docs.

Examples:

files.template(
    {'Create a templated file'},
    'templates/somefile.conf.j2',
    '/etc/somefile.conf',
)

files.template(
    {'Create service file'},
    'templates/myweb.service.j2',
    '/etc/systemd/system/myweb.service',
    mode='755',
    user='root',
    group='root',
)

# Example showing how to pass python variable to template file.
# The .j2 file can use `{{ foo_variable }}` to be interpolated.
foo_variable = 'This is some foo variable contents'
files.template(
    {'Create a templated file'},
    'templates/foo.j2',
    '/tmp/foo',
    foo_variable=foo_variable,
)