gpfs

Wrappers to call GPFS commands

exception accre.gpfs.GPFSCommandError

Bases: ACCREError

An error occurred running a GPFS command

accre.gpfs.get_filesystem_info(filesystem=None)

Runs mmlsfs command, which returns various attributes of the filesystem being queried (execute “man mmlsfs” from any box with GPFS installed for full details).

Parameters:

filesystem (str) – the GPFS filesystem to query

Returns:

dictionary of filesystem attributes and their values

accre.gpfs.get_filesystem_quotas(quota_type, quota_target, filesystem=None)

Runs mmlsquota command, which returns quota information for the specified user, group, or fileset as appropriate (execute “man mmlsquota” from any box with GPFS installed for full details).

Parameters:

• quota_type (str) – -g for group, -j for fileset, -u for user

• quota_target (str) – group, fileset, or user to be looked up

• (optional) (str filesystem) – only return info for specified fs

Returns:

dict keyed by fileset where each value is a dict of quota information for that fileset

accre.gpfs.get_gpfs_filesystems()

Gets GPFS filesystems by reading /etc/fstab … mount type (3rd field) will be “gpfs” and the actual filesystem is determined by the “dev=” field in the mount options.

Returns:

two lists - local filesystems and remote filesystems

accre.gpfs.get_longest_waiter()

Retrieve the length in seconds, reason, and node (if applicable) of the longest waiter present on this node.

Returns:

dictionary with fields length, reason, and node containing information about the longest waiter on this node. If there are no waiters all fields will be set to None.

accre.gpfs.get_quota_usage(filesystem, quota_type, timeout=600)

Return a list of quota/usage records for the specified GPFS filesytem and quota type that is taken by parsing the result of the mmrepquota command. This command should be run on a GPFS nsd or manager node. The formatting of the result lists block usage in bytes and is suitable for ingestion into the GFPS_USAGE table of the admin database, see accre.database.

Parameters:

• filesystem (str) – The GPFS filesystem to query

• quota_type (str) – either USR, GRP, or FILESET

• timeout (int) – Timeout in seconds to run the command, note that it may take a while so the default is 10 minutes

Returns:

list or quota/usage records

Return type:

list(dict)

accre.gpfs.parse_gpfs_usage(raw, filesystem=None)

Takes the output of mmrepquota with –block-size set to auto and returns a list of dictionaries of quota/usage records. The formatting of the result lists block usage in bytes and is suitable for ingestion into the GFPS_USAGE table of the admin database, see accre.database.

Parameters:

• raw (str) – Output of mmrepquota decoded as utf-8

• filesystem (str) – Name of filesystem to insert into records

Returns:

Quota usage records

Return type:

list(dict)

accre.gpfs.parse_gpfs_y_command(raw)

Parse the result of the raw output of GPFS cli command executed with the -Y option into a list of dictionaries keyed according to the header line in the output. ‘reserved’ fields are discarded.

The first three fields in each line of the raw output appear to be special. The first is the name of the command and is set as a “command” field. The second is the type of entity and will be set to a “type” field. The third field is HEADER if the line is defines all further fields for an entity or is 0 if it is a regular output.

The HEADER entries are first parsed to determine the structure of each entry and are not added to the return list.

Empty or reserved fields are discarded.

Parameters:

raw (str) – raw output of the GPFS command with -Y

Returns:

List of dictionaries containing parsed output

Return type:

list(dict(str, str))

accre.gpfs.run_gpfs_command(arglist, ssh=False, cluster=None, testcluster=False, timeout=60)

Runs a GPFS command as a subprocess and returns the standard output as a string decoded from utf-8. Optionally ssh out to a configured GPFS node and use a timeout.

Parameters:

• arglist (list(str)) – List of arguments run, including command name

• ssh (bool) – If True, ssh to the configured gpfs node to run command

• cluster (str) – If True and if ssh is True, run on a configured node in the specified cluster, (i.e. accre or accre2)

• testcluster (bool) – If True and if ssh is True, run on a configured test cluster node rather than the production cluster

• timeout (int) – Maximum time in seconds to wait for command to complete

Returns:

Standard output of command interpreted as utf-8 text

Return type:

str

accre.gpfs.run_mmlslicense(ssh=False, testcluster=False)

Run the GPFS ‘mmlslicense’ command and return output as a dictionary of node counts per license type or missing license type.

Parameters:

• ssh (bool) – If True, ssh out to the configured node running gpfs to run the command. If False, run locally.

• testcluster (bool) – If True and if ssh is also true, ssh out to the configured node running GPFS on the test cluster.

Returns:

GPFS license counts

Return type:

dict(str, int)

accre.gpfs.run_mmlsnode(ssh=False, testcluster=False)

Run the GPFS ‘mmlsnode’ command and return output as a dictionary keyed by cluster with a list of nodes as the values.

Parameters:

• ssh (bool) – If True, ssh out to the configured node running gpfs to run the command. If False, run locally.

• testcluster (bool) – If True and if ssh is also true, ssh out to the configured node running GPFS on the test cluster.

Returns:

GPFS nodes by cluster

Return type:

dict(str, list(str))

accre.gpfs.set_fileset_quota(filesystem, fileset, block_quota, block_limit, file_quota=None, file_limit=None, ssh=False, cluster='accre2')

Set the soft block quota and limit for the specified fileset and optionally set the file quota and limit

Parameters:

• filesystem (str) – Filesystem that the fileset belongs to

• fileset (str) – Name of the GPFS fileset

• block_quota (str) – Soft block quota (should be in units of k, M, G, T, etc..)

• block_limit (str) – Hard block quota/limit (should be in units of k, M, G, T, etc..)

• file_quota (str) – Soft file quota, or no quota change if None

• file_limit (str) – Soft file limit, or no quota change if None

• ssh (bool) – If True, ssh out to the configured node running gpfs to run the command. If False, run locally.

• cluster (str) – GPFS cluster that contains the fileset, defaults to accre2. You cannot set quotas on a remote cluster so the node running this function or sshed to must be in the same cluster as the filesystem.