database.client.groups

Mixin class for databse client with group related functions

class accre.database.client.groups.DBClientGroupMixin[source]

Bases: object

Functionality related to ACCRE groups

activate_group(name, reason=None, log_date=None)[source]

(Re)Activate the group in the Database. Note that this does not activate a group in LDAP, but simply marks a group as one that is currently used on the cluster. A reason string is required.

Parameters:

  • name (str) – Group to be activated

  • reason (str) – Reason for activation. Cite a ticket if possible.

  • log_date (datetime.datetime) – Date to use for the log entry, defaults to the current date.

add_group(*, name, group_id, formatted_name, pi, posix=True, scheduler=True, account='guest', fairshare=1, qos='normal', max_cpu=16, max_mem=None, max_runmins=None, partitions=('debug', 'production'), active=True, join_date=None, user_primary=True)[source]

Add a new laboratory or other group to the database.

If an active group already exists with the same group_id, then an exception is raised.

Parameters:

  • name (str) – Name of the group (POSIX)

  • group_id (int) – Numeric GID for the group

  • formatted_name (str) – Full name for the group

  • pi (str) – VUNetID of the group principle investigator. This PI must already exist in the database.

  • account (str) – The name of a billable account. This account must already exist in the database (default guest).

  • posix (bool) – True if this group should exist as a POSIX group in LDAP

  • scheduler (bool) – True if this group should exist as a group in the cluster scheduler

  • fairshare (int) – Scheduler fairshare for the group

  • qos (str) – Scheduler QOS for the group

  • max_cpu (int) – Scheduler group CPU core maximum

  • max_mem (str) – Scheduler group memory maximum, should indicate units if present, i.e. 120G or 10000M

  • max_runmins (int) – Scheduler group running minutes maximum

  • partitions (list(str)) – Scheduler partitions for which group members should have corresponding associations. These must already exist in the database.

  • active (bool) – True if the group is active on the cluster

  • join_date (datetime.datetime) – Date that the group joined the cluster, defaults to the current date

  • user_primary (bool) – True if this group can be a primary group for normal users, i.e. should be an option on account creation.

add_group_acc_partition(name, partition, gpus, log_date=None)[source]

Add a new scheduler accelerated partition to a group

Parameters:

  • name (str) – group to add partition to

  • partition (str) – Paritition to add to the grouo

  • gpus (int) – Quantity of GPUs allowed for bursting in this partition for this group.

add_group_affiliate(name, affiliate, log_date=None, reason=None)[source]

Add specified affiliate group to a group

Parameters:

  • name (str) – group to add the affiliate to

  • log_date (datetime.datetime) – Date to use for the log entry, defaults to the current date.

  • reason (str) – explanation of the change for logging

Paran str affiliate:

affiliate group to add

add_group_disclosure(*, group, pi_vunetid, pi_name, pi_email, data_classification, preferred_group_name, disclosure_data, delegate_vunetid=None, delegate_name=None, delegate_email=None, department=None, usage_description=None, creation_time=None, renewal_time=None)[source]

Create a new ACCRE group disclosure with the specified attributes.

Parameters:

  • group (str) – The group for which this disclosure applies, which must already be present in the database

  • pi_vunetid (str) – The VUNetID of the current group PI making creating the new disclosure.

  • pi_name (str) – The full name of the current group PI making creating the new disclosure.

  • pi_email (str) – The email of the current group PI making creating the new disclosure.

  • data_classification (int) – Positive integer 1-4 representing the classification level of the group data.

  • preferred_group_name (str) – Chosen group name from the PI which may need to be modified to ACCRE standards/policies

  • disclosure_data (dict) – a dict representing a valid JSON object with legal/other information for the disclosure, such as acknowledgement/understanding booleans and text fields for additonal information that may change over time

  • delegate_vunetid (str) – (Optional) The VUNetID of a group member to be authorized as a trusted group manager.

  • delegate_name (str) – (Optional) The full name of a group member to be authorized as a trusted group manager.

  • delegate_email (str) – (Optional) The email of a group member to be authorized as a trusted group manager.

  • deparment (str) – (Optional) The university department of the group

  • usage_description (str) – (Optional

  • creation_time (datetime.datetime) – Time when this disclosure was digitally signed, if set to None it will be the current time

  • renewal_time (datetime.datetime) – Time when this disclosure was renewed, if set to None it will be the current time

add_group_log(name, comment, log_date=None)[source]

Add an entry to the given group’s log.

Parameters:

  • group (str) – name of the group

  • comment (str) – entry to add to the log

  • log_date (datetime.datetime) – Date to use for the log entry, defaults to the current date.

add_group_managers(group, *managers, log_date=None)[source]

Add the given trusted manager (identified by vunetid) to the group.

Parameters:

  • group (str) – Name of the group (must be in the GROUPS table)

  • manager (str) – vunetid of the trusted manager (must be in the PIS

table)

add_group_partitions(name, *partitions, log_date=None, fairshare=None, qos=None, max_cpu=None, max_mem=None, max_runmins=None)[source]

Add specified scheduler partitions to a group

Parameters:

  • name (str) – group to add partition(s) to

  • partitions (str) – Parititions to add to the grouo

  • fairshare (int) – A custom fairshare to set for associations generated by this partition record if not None

  • qos (str) – A custom qos to set for associations generated by this partition record if not None

  • max_cpu (int) – A custom cpu restriction to set for associations generated by this partition record if not None

  • max_mem (str) – A custom memory restriction to set for associations generated by this partition record if not None

  • max_runmins (str) – A custom runtime restriction to set for associations generated by this partition record if not None

  • log_date (datetime.datetime) – Date to use for the log entry, defaults to the current date.

all_groups_info(active=False, posix=False, scheduler=False, limit=None, offset=None)[source]

Retrieve general information for an ACCRE group including members of the group and scheduler partitions by default Note that partitions and members will not be returned

Parameters:

  • active (bool) – List only active groups if set to true

  • posix (bool) – List only posix groups if set to true

  • scheduler (bool) – List only scheduler groups if set to true

  • limit (int) – Maximum number of records to return, return all if None

  • offset (int) – Offset from first record to begin list of records, start at the first record if None

Returns:

General group information for all groups

Return type:

list(dict)

deactivate_group(name, reason=None, log_date=None)[source]

Deactivate the group in the Database. Note that this does not remove a user in LDAP, but simply marks a group as one that is no longer used on the cluster. A reason string is required.

Parameters:

  • name (str) – Group to be deactivated

  • reason (str) – Reason for deactivation. Cite a ticket if possible.

  • log_date (datetime.datetime) – Date to use for the log entry, defaults to the current date.

get_group_logs(name)[source]

Fetch group logs for a given group sorted by newest first

Parameters:

name (str) –

Returns:

list of all group log dates and comments for the vunetid

Return type:

list(tuple(datetime.datetime, comment))

group_by_gid(gid)[source]

Return the name of the group specified by gid or throw an exception if the group does not exist

Parameters:

gid (int) – cluster group numerical gid to be queried

Returns:

group name

Return type:

str

group_exists(name)[source]

Return true if the specified cluster group exists in the database

Parameters:

name (str) – cluster group to be queried

Returns:

True if the group is found in the database

Return type:

bool

group_info(name, members=True, active_members=True, partitions=True, partition_details=False, acc_partitions=True, acc_partition_details=False, parent_account_details=False, affiliates=True, gpfs_filesets=False, managers=True, disclosure=False, custom_servers=False)[source]

Retrieve general information for an ACCRE group including members of the group and scheduler partitions by default

Parameters:

  • name (str) – ACCRE group to be queried

  • members (bool) – Show members of the group

  • active_members (bool) – Show active members of the group if set to true

  • partitions (bool) – Show scheduler partitions for the group if set to true

  • partition_details (bool) – Show scheduler partition information (QOS, restrictions) for each paritition in a partition_details field if true. This will be a list of dicts, one for each partition.

  • acc_partitions (bool) – Show accelerated scheduler partitions for the group if set to true

  • acc_partition_details (bool) – Show accelerated scheduler partition information (gpus) for each paritition in an acc_partition_details field if true. This will be a list of dicts, one for each accelerated partition.

  • parent_account_details (bool) – Show fairshare, qos, and limits of the parent account in slurm, if True

  • affiliates (bool) – Show a list of affiliate groups if True

  • gpfs_filesets (bool) – If true, provide a list of active GPFS filesets owned by the group as a list of tuples (filesystem, name, fileset)

  • managers (bool) – If true, show a list of trusted managers for this group, if any.

  • disclosure (bool) – If true, add the group disclosure data if one exists, or otherwise set a disclosure key to a value of None

  • custom_servers (bool) – If true, add a list of custom servers owned by the group

Returns:

General group information

Return type:

dict

list_groups(active=False, posix=False, scheduler=False)[source]

List all groups in the database, or all active groups.

Parameters:

  • active (bool) – List only active groups if set to true

  • posix (bool) – List only posix groups if set to true

  • scheduler (bool) – List only scheduler groups if set to true

Returns:

List of group names

Return type:

list(str)

list_groups_by_account()[source]

List all active groups organized into a dictionary keyed by parent account with values of lists of groups under that business account.

Returns:

dict of groups by business account

Return type:

dict(str, list(str)))

list_user_primary_groups()[source]

Return a list of tuples of group names and formatted names for active groups that are suitable as a primary group for a normal user, i.e. should be selectable as a group for a new account, via the user_primary column.

Returns:

list of group names and formatted names

Return type:

list(tuple(str, str))

modify_group(group, reason=None, log_date=None, **updates)[source]

Modify one or more fields of an existing ACCRE group and log the change with an optional reason.

Parameters:

  • group (str) – Name of the group

  • reason (str) – Explanation of the change to be logged

  • updates (dict) – Columns to be updated with their values

  • log_date (datetime.datetime) – Date to log the modification, defaults to the current date

modify_group_acc_partition(group, partition, gpus, reason=None, log_date=None)[source]

Modify the GPU quantity of an existing ACCRE group-acc-partition association log the change for the group with an optional reason.

Parameters:

  • group (str) – Name of the group

  • partition (str) – Name of the partition

  • gpus (int) – New quantity for GPU limit

  • reason (str) – Explanation of the change to be logged

  • log_date (datetime.datetime) – Date to log the modification, defaults to the current date

modify_group_partition(group, partition, reason=None, log_date=None, **updates)[source]

Modify one or more fields of an existing ACCRE group-partition association log the change for the group with an optional reason.

Parameters:

  • group (str) – Name of the group

  • partition (str) – Name of the partition

  • reason (str) – Explanation of the change to be logged

  • updates (dict) – Columns to be updated with their values

  • log_date (datetime.datetime) – Date to log the modification, defaults to the current date

remove_group_acc_partition(name, partition, log_date=None)[source]

Remove specified scheduler accelerated partition from a group

Parameters:

  • name (str) – group to add partition to

  • partition (str) – partition to remove from the group

  • log_date (datetime.datetime) – Date to use for the log entry, defaults to the current date.

remove_group_affiliate(name, affiliate, log_date=None, reason=None)[source]

Remove specified affiliate group from a group

Parameters:

  • name (str) – group to remove the affiliate from

  • affiliate (str) – affiliate group to remove

  • log_date (datetime.datetime) – Date to use for the log entry, defaults to the current date.

  • reason (str) – explanation of the change for logging

remove_group_managers(group, *managers, log_date=None)[source]

Remove the given trusted manager from the group.

Parameters:

  • group (str) – Name of the group (must be in the GROUPS table)

  • manager (str) – vunetid of the trusted manager (must be in the PIS

table)

remove_group_partitions(name, *partitions, log_date=None)[source]

Remove specified scheduler partitions from a group

Parameters:

  • name (str) – group to add partition(s) to

  • partitions (str) – partitions to remove from the group

  • log_date (datetime.datetime) – Date to use for the log entry, defaults to the current date.

renew_group_disclosure(group, ticket=None, **updates)[source]

Modify one or more fields of an existing ACCRE group and log the change indicating the ticket if present

Parameters:

  • group (str) – Name of the group

  • ticket (str) – RT ticket number corresponding to the renewal, if a ticket exists

  • updates (dict) – Disclosure columns to be updated with their values