database.client.gpfs

Mixin class for databse client with GPFS related functions

class accre.database.client.gpfs.DBClientGPFSMixin[source]

Bases: object

Functionality related to GPFS

add_gpfs_fileset(*, filesystem, name, block_quota, block_limit, file_quota, file_limit, gid, path, user_path, active=True, join_date=None, fileset='root')[source]

Add a new GPFS fileset record determined by a (filesystem, name, fileset) identifier.

Note that generally in the current (summer 2020) setup the “fileset” will be “root”. These records are intended to track filesets owned by ACCRE groups on /data or /dors, not anything relating to user home or scratch quotas.

Parameters:

  • filesystem (str) – GPFS filesystem record refers to

  • name (str) – Fileset identifier

  • fileset (str) – Parent fileset, typically root

  • block_quota (int) – (soft) quota in bytes

  • block_limit (int) – (hard quota) limit in bytes

  • file_quota (int) – (soft) quota in number of files

  • file_limit (int) – (hard quota) limit in number of files

  • gid (int) – Numeric GID of the group that owns the fileset

  • path (str) – The path that the fileset is linked to in GPFS

  • user_path (str) – The path that the user should access the fileset by, either via symlink, bind mount, or other method

  • active (bool) – If this fileset should still exist on the cluster

  • join_date (datetime.datetime) – Date the filesystem added to the cluster, defaults to the current date

add_gpfs_quota_contact(entity, etype, email)[source]

Add the quota contact email for the given entity and type (group, fileset).

Parameters:

  • entity (str) – Name of the group or fileset being queried

  • etype (str) – Type of gpfs quota entity (fileset, group)

  • email (str) – quota contact email address

all_gpfs_fileset_info(active=True)[source]

Retrieve general information about all active filesets, or optionall all filesets in the database

Parameters:

active (bool) – GPFS filesystem record refers to

Returns:

General fileset information for all filesets

Return type:

list(dict)

get_all_gpfs_quota_contacts()[source]

Retrieve all quota contact emails for all entities of all types (group, fileset, etc.) as a list of dictionaries, one for each entry in the quota contacts table.

Returns:

List of quota contact entries

Return type:

list(dict(str,str))

get_gpfs_longest_waiters(since=None)[source]

Retrieve the longest waiter for each host in the database where the longest waiter has a set length sorted in decreasing order by length. If since is set to a datetime, only retrieve hosts that have been updated since that datetime.

Parameters:

since (datetime.datetime) – Earliest update time for which to retrieve managed host records

Returns:

Managed inventory information for all hosts with a valid waiter (length is not null)

Return type:

list(dict)

get_gpfs_quota_contacts(entity, etype, fallback=False)[source]

Retrieve all quota contact emails for the given entity of the given type (group, fileset). If there are no quota contact entries in the database, and a group exists with the same name as the entity, then the PI email for that group is returned in the list.

Parameters:

  • entity (str) – Name of the group or fileset being queried

  • etype (str) – Type of gpfs quota entity (fileset, group)

  • fallback (bool) – If True, and a group with the same name as entity exists, return the PI email for that group if there are no quota contacts.

Returns:

List of quota contact emails for the entity

Return type:

list(str)

get_gpfs_usage_records(*, filesystem, type, name=None, fileset=None)[source]

Retrieve all GPFS usage records for the given filesystem, and type for all filesets and names or for a specific fileset and/or name if specified.

Parameters:

  • filesystem (str) – The GPFS filesystem to query

  • type (str) – Either USR, GRP, or FILESET

  • name (str) – Record name if specified (None for any name)

  • fileset (str) – Fileset name if specified (None for any fileset)

Returns:

Usage and quota information for all fileset records specified.

Return type:

list(dict)

gpfs_fileset_info(*, filesystem, name, fileset='root', usage=False)[source]

Retrieve general information for a GPFS fileset. If usage is set to True, also look up most recent usage record for the fileset and give the time of the last usage check. Usage will be set to None if the there is no usage record available in the database

Parameters:

  • filesystem (str) – GPFS filesystem record refers to

  • name (str) – Fileset identifier

  • fileset (str) – Parent fileset, typically root

Returns:

General fileset information

Return type:

dict

modify_gpfs_fileset(*, filesystem, name, fileset='root', **updates)[source]

Modify one or more fields of an existing GPFS fileset. Note that this method does not log the change. If the associated group exists, it is suggested to add a log entry to that group.

Parameters:

  • filesystem (str) – GPFS filesystem record refers to

  • name (str) – Fileset identifier

  • fileset (str) – Parent fileset, typically root

  • updates (dict) – Columns to be updated with their values

remove_all_gpfs_quota_contacts(entity, etype)[source]

Remove all quota contact emails for the given entity and type (group, fileset).

Parameters:

  • entity (str) – Name of the group or fileset being queried

  • etype (str) – Type of gpfs quota entity (fileset, group)

remove_gpfs_quota_contact(entity, etype, email)[source]

Remove the quota contact email for the given entity and type (group, fileset).

Parameters:

  • entity (str) – Name of the group or fileset being queried

  • etype (str) – Type of gpfs quota entity (fileset, group)

  • email (str) – quota contact email address

update_gpfs_long_waiter(*, hostname, last_check, length=None, reason=None, node=None)[source]

Insert or update a GPFS long waiter record for a given hostname

Parameters:

  • hostname (str) – Internal hostname of the server (required)

  • last_check (datetime.datetime) – Last time this node was checked for long waiters (required)

  • length (float) – Length of the longest waiter in seconds

  • reason (str) – Reason for the waiter

  • node (str) – IP address of the node the waiter is for

update_gpfs_usage(*, filesystem, name, fileset, type, block_usage, block_quota, block_limit, block_grace, file_usage, file_quota, file_limit, file_grace)[source]

Insert or update a GPFS usage record for a given (filesystem, name, fileset) identifier.

Parameters:

  • filesystem (str) – GPFS filesystem record refers to

  • name (str) – Fileset, user, or group identifier

  • fileset (str) – Parent or associated fileset

  • type (str) – Either FILESET, USR, or GRP

  • block_usage (int) – usage in bytes

  • block_quota (int) – (soft) quota in bytes

  • block_limit (int) – (hard quota) limit in bytes

  • block_grace (string) – Grace period for size quota, could be none, expired, or a GPFS-format time value

  • file_usage (int) – usage in number of files

  • file_quota (int) – (soft) quota in number of files

  • file_limit (int) – (hard quota) limit in number of files

  • file_grace (string) – Grace period for file count quota, could be none, expired, or a GPFS-format time value