AWS Driver

If using the AWS driver to upload diskimages, see VM Import/Export service role for information on configuring the required permissions in AWS. You must also create an S3 Bucket for use by Nodepool if uploading images (except when using the ebs-direct upload method).

Selecting the aws driver adds the following options to the providers section of the configuration.

providers.[aws]
providers.[aws].name
providers.[aws].region-name
providers.[aws].profile-name
providers.[aws].rate
providers.[aws].boot-timeout
providers.[aws].launch-timeout
providers.[aws].max-cores
providers.[aws].max-servers
providers.[aws].max-ram
providers.[aws].max-resources
providers.[aws].launch-retries
providers.[aws].post-upload-hook
providers.[aws].object-storage
providers.[aws].image-format
providers.[aws].image-import-timeout
providers.[aws].cloud-images
providers.[aws].diskimages
providers.[aws].pools

providers.[aws] Type: list

An AWS provider’s resources are partitioned into groups called pool (see providers.[aws].pools for details), and within a pool, the node types which are to be made available are listed (see providers.[aws].pools.labels for details).

See Boto Configuration for information on how to configure credentials and other settings for AWS access in Nodepool’s runtime environment.

Note

For documentation purposes the option names are prefixed providers.[aws] to disambiguate from other drivers, but [aws] is not required in the configuration (e.g. below providers.[aws].pools refers to the pools key in the providers section when the aws driver is selected).

Example:

providers:
  - name: ec2-us-west-2
    driver: aws
    region-name: us-west-2
    cloud-images:
      - name: debian9
        image-id: ami-09c308526d9534717
        username: admin
    pools:
      - name: main
        max-servers: 5
        subnet-id: subnet-0123456789abcdef0
        security-group-id: sg-01234567890abcdef
        labels:
          - name: debian9
            cloud-image: debian9
            instance-type: t3.medium
            iam-instance-profile:
              arn: arn:aws:iam::123456789012:instance-profile/s3-read-only
            key-name: zuul
            tags:
              key1: value1
          - name: debian9-large
            cloud-image: debian9
            instance-type: t3.large
            key-name: zuul
            use-spot: True
            tags:
              key1: value1
              key2: value2

providers.[aws].name (required): A unique name for this provider configuration.

providers.[aws].region-name (required): Name of the AWS region to interact with.

providers.[aws].profile-name

The AWS credentials profile to load for this provider. If unspecified the boto3 library will select a profile.

See Boto Configuration for more information.

providers.[aws].rate Default: 2.0 Type: float: The number of operations per second to perform against the provider.

providers.[aws].boot-timeout Default: 180 Type: int seconds: Once an instance is active, how long to try connecting to the image via SSH. If the timeout is exceeded, the node launch is aborted and the instance deleted.

providers.[aws].launch-timeout Default: 3600 Type: int seconds: The time to wait from issuing the command to create a new instance until that instance is reported as “active”. If the timeout is exceeded, the node launch is aborted and the instance deleted.

providers.[aws].max-cores Default: unlimited Type: int: Maximum number of cores usable from this provider’s pools by default.

providers.[aws].max-servers Default: unlimited Type: int: Maximum number of servers spawnable from this provider’s pools by default.

providers.[aws].max-ram Default: unlimited Type: int: Maximum RAM usable from this provider’s pools by default.

providers.[aws].max-resources Default: unlimited Type: dict

A dictionary of other quota resource limits. AWS has quotas for certain instance types. These may be specified here to limit Nodepool’s usage.

The following example limits the number of high-memory instance cores:

max-resources:
  'L-43DA4232': 448

See instance quotas for more information.

providers.[aws].launch-retries Default: 3: The number of times to retry launching a node before considering the request failed.

providers.[aws].post-upload-hook Default: None Type: string

Filename of an optional script that can be called after an image has been uploaded to a provider but before it is taken into use. This is useful to perform last minute validation tests before an image is really used for build nodes. The script will be called as follows:

<SCRIPT> <PROVIDER> <EXTERNAL_IMAGE_ID> <LOCAL_IMAGE_FILENAME>

If the script returns with result code 0 it is treated as successful otherwise it is treated as failed and the image gets deleted.

providers.[aws].object-storage

This section is only required when using Nodepool to upload diskimages.

providers.[aws].object-storage.bucket-name: The name of a bucket to use for temporary storage of diskimages while creating snapshots. The bucket must already exist.

providers.[aws].image-format Default: raw Type: str: The image format that should be requested from diskimage-builder and also specified to AWS when importing images. One of: ova, vhd, vhdx, vmdk, raw (not all of which are supported by diskimage-builder).

providers.[aws].image-import-timeout Type: int: Generally there is no limit on the amount of time a successful image import can take. However, some import tasks may encounter temporary resource limitations from AWS. In these cases, if this value is set, Nodepool will retry the import tasks until the timeout is reached. If this is unset (the default), then the first resource limitation detected will result in an error. The value is in seconds.

providers.[aws].cloud-images Type: list

Each entry in this section must refer to an entry in the labels section.

cloud-images:
  - name: ubuntu1804
    image-id: ami-082fd9a18128c9e8c
    username: ubuntu
  - name: ubuntu1804-by-filters
    image-filters:
      - name: name
        values:
         - named-ami
    username: ubuntu
  - name: my-custom-win2k3
    connection-type: winrm
    username: admin

Each entry is a dictionary with the following keys

providers.[aws].cloud-images.name (required) Type: string: Identifier to refer this cloud-image from providers.[aws].pools.labels section. Since this name appears elsewhere in the nodepool configuration file, you may want to use your own descriptive name here and use image-id to specify the cloud image so that if the image id changes on the cloud, the impact to your Nodepool configuration will be minimal. However, if image-id is not provided, this is assumed to be the image id in the cloud.

providers.[aws].cloud-images.image-id Type: str: If this is provided, it is used to select the image from the cloud provider by ID. Either this field or providers.[aws].cloud-images.image-filters must be provided.

providers.[aws].cloud-images.image-filters Type: list

If provided, this is used to select an AMI by filters. If the filters provided match more than one image, the most recent will be returned. Either this field or providers.[aws].cloud-images.image-id must be provided.

Each entry is a dictionary with the following keys

providers.[aws].cloud-images.image-filters.name (required) Type: str: The filter name. See Boto describe images for a list of valid filters.

providers.[aws].cloud-images.image-filters.values (required) Type: list: A list of string values on which to filter.

providers.[aws].cloud-images.username Type: str: The username that a consumer should use when connecting to the node.

providers.[aws].cloud-images.python-path Default: auto Type: str: The path of the default python interpreter. Used by Zuul to set ansible_python_interpreter. The special value auto will direct Zuul to use inbuilt Ansible logic to select the interpreter on Ansible >=2.8, and default to /usr/bin/python2 for earlier versions.

providers.[aws].cloud-images.connection-type Type: str: The connection type that a consumer should use when connecting to the node. For most images this is not necessary. However when creating Windows images this could be ‘winrm’ to enable access via ansible.

providers.[aws].cloud-images.connection-port Default: 22/ 5986 Type: int: The port that a consumer should use when connecting to the node. For most diskimages this is not necessary. This defaults to 22 for ssh and 5986 for winrm.

providers.[aws].cloud-images.shell-type Default: sh Type: str

The shell type of the node’s default shell executable. Used by Zuul to set ansible_shell_type. This setting should only be used

For a windows image with the experimental connection-type ssh in which case cmd or powershell should be set and reflect the node’s DefaultShell configuration.
If the default shell is not Bourne compatible (sh), but instead e.g. csh or fish, and the user is aware that there is a long-standing issue with ansible_shell_type in combination with become.

providers.[aws].diskimages Type: list

Each entry in a provider’s diskimages section must correspond to an entry in diskimages. Such an entry indicates that the corresponding diskimage should be uploaded for use in this provider. Additionally, any nodes that are created using the uploaded image will have the associated attributes (such as flavor or metadata).

If an image is removed from this section, any previously uploaded images will be deleted from the provider.

diskimages:
  - name: bionic
    pause: False
  - name: windows
    connection-type: winrm
    connection-port: 5986

Each entry is a dictionary with the following keys

providers.[aws].diskimages.name (required) Type: string: Identifier to refer this image from providers.[aws].pools.labels and diskimages sections.

providers.[aws].diskimages.pause Default: False Type: bool: When set to True, nodepool-builder will not upload the image to the provider.

providers.[aws].diskimages.username Type: str: The username that should be used when connecting to the node.

Warning

This option is deprecated. Specify the username on the diskimage definition itself instead.

providers.[aws].diskimages.connection-type Type: string: The connection type that a consumer should use when connecting to the node. For most diskimages this is not necessary. However when creating Windows images this could be winrm to enable access via ansible.

providers.[aws].diskimages.connection-port Default: 22 / 5986 Type: int: The port that a consumer should use when connecting to the node. For most diskimages this is not necessary. This defaults to 22 for ssh and 5986 for winrm.

providers.[aws].diskimages.python-path Default: auto Type: str: The path of the default python interpreter. Used by Zuul to set ansible_python_interpreter. The special value auto will direct Zuul to use inbuilt Ansible logic to select the interpreter on Ansible >=2.8, and default to /usr/bin/python2 for earlier versions.

providers.[aws].diskimages.shell-type Default: sh Type: str

The shell type of the node’s default shell executable. Used by Zuul to set ansible_shell_type. This setting should only be used

For a windows image with the experimental connection-type ssh in which case cmd or powershell should be set and reflect the node’s DefaultShell configuration.
If the default shell is not Bourne compatible (sh), but instead e.g. csh or fish, and the user is aware that there is a long-standing issue with ansible_shell_type in combination with become.

providers.[aws].diskimages.architecture Default: x86_64 Type: str: The architecture of the image. See the AWS RegisterImage API documentation for valid values.

providers.[aws].diskimages.ena-support Default: True Type: bool: Whether the image has support for the AWS Enhanced Networking Adapter (ENA). Many newer operating systems include driver support as standard and some AWS instance types require it.

providers.[aws].diskimages.volume-type Default: gp3 Type: str: The root EBS volume type for the image. Only used with the snapshot or ebs-direct import methods.

providers.[aws].diskimages.volume-size Type: int: The size of the root EBS volume, in GiB, for the image. If omitted, the volume size reported for the imported snapshot will be used. Only used with the snapshot or ebs-direct import methods.

providers.[aws].diskimages.imds-support Type: str: To enforce usage of IMDSv2 by default on instances created from the image, set this value to v2.0. If omitted, IMDSv2 is optional by default. This is only supported using the snapshot or ebs-direct import methods.

providers.[aws].diskimages.import-method Default: snapshot

The method to use when importing the image.

snapshot: This method uploads the image file to AWS as a snapshot and then registers an AMI directly from the snapshot. This is faster compared to the image method and may be used with operating systems and versions that AWS does not otherwise support. However, it is incompatible with some operating systems which require special licensing or other metadata in AWS.

ebs-direct: This is similar to the snapshot method, but uses the EBS direct API instead of S3. This may be faster and more efficient, but it may incur additional costs.

image: This method uploads the image file to AWS and performs an “image import” on the file. This causes AWS to boot the image in a temporary VM and then take a snapshot of that VM which is then used as the basis of the AMI. This is slower compared to the snapshot method and may only be used with operating systems and versions which AWS already supports. This may be necessary in order to use Windows images.

providers.[aws].diskimages.iops Type: int: The number of I/O operations per second to be provisioned for the volume. The default varies based on the volume type; see the documentation under EBS volume type for the specific volume type for details.

providers.[aws].diskimages.throughput Type: int: The throughput of the volume in MiB/s. This is only valid for gp3 volumes.

providers.[aws].diskimages.tags Default: None Type: dict: A dictionary of tags to add to uploaded images. This will be merged with any existing metadata from the global diskimage configuration for this image. Avoid the use of nodepool_ as a key prefix since Nodepool uses this for internal values.

providers.[aws].pools Type: list

A pool defines a group of resources from an AWS provider. Each pool has a maximum number of nodes which can be launched from it, along with a number of cloud-related attributes used when launching nodes.

providers.[aws].pools.name (required): A unique name within the provider for this pool of resources.

providers.[aws].pools.availability-zone Type: str: If provided, instances launched from this pool will be assigned to the specified availibility zone. If omitted, AWS will select from the available zones.

providers.[aws].pools.priority Default: 100 Type: int

The priority of this provider pool (a lesser number is a higher priority). Nodepool launchers will yield requests to other provider pools with a higher priority as long as they are not paused. This means that in general, higher priority pools will reach quota first before lower priority pools begin to be used.

This setting may be specified at the provider level in order to apply to all pools within that provider, or it can be overridden here for a specific pool.

providers.[aws].pools.node-attributes Type: dict: A dictionary of key-value pairs that will be stored with the node data in ZooKeeper. The keys and values can be any arbitrary string.

providers.[aws].pools.max-cores Type: int: Maximum number of cores usable from this pool. Defaults to providers.[aws].max-cores.

providers.[aws].pools.max-servers Type: int: Maximum number of servers spawnable from this pool. Defaults to providers.[aws].max-servers.

providers.[aws].pools.max-ram Type: int: Maximum RAM usable from this pool. Defaults to providers.[aws].max-ram.

providers.[aws].pools.max-resources Type: dict

A dictionary of other quota resource limits. AWS has quotas for certain instance types. These may be specified here to limit Nodepool’s usage. Defaults to providers.[aws].max-resources.

The following example limits the number of high-memory instance cores:

max-resources:
  'L-43DA4232': 448

See instance quotas for more information.

providers.[aws].pools.subnet-id: If provided, specifies the subnet to assign to the primary network interface of nodes.

providers.[aws].pools.security-group-id: If provided, specifies the security group ID to assign to the primary network interface of nodes.

providers.[aws].pools.public-ip-address Default: True Type: bool: Deprecated alias for providers.[aws].pools.public-ipv4.

providers.[aws].pools.public-ipv4 Default: True Type: bool: Specify if a public IPv4 address shall be attached to nodes.

providers.[aws].pools.public-ipv6 Default: True Type: bool: Specify if a public IPv6 address shall be attached to nodes.

providers.[aws].pools.use-internal-ip Default: false Type: bool: If a public IP is attached but Nodepool should prefer the private IP, set this to true.

providers.[aws].pools.host-key-checking Default: True Type: bool: Whether to validate SSH host keys. When true, this helps ensure that nodes are ready to receive SSH connections before they are supplied to the requestor. When set to false, nodepool-launcher will not attempt to ssh-keyscan nodes after they are booted. Disable this if nodepool-launcher and the nodes it launches are on different networks, where the launcher is unable to reach the nodes directly, or when using Nodepool with non-SSH node platforms. The default value is true.

providers.[aws].pools.labels Type: list

Each entry in a pool’s labels section indicates that the corresponding label is available for use in this pool. When creating nodes for a label, the flavor-related attributes in that label’s section will be used.

labels:
  - name: bionic
    instance-type: m5a.large

Each entry is a dictionary with the following keys

providers.[aws].pools.labels.name (required)
Type: str

Identifier to refer to this label.

providers.[aws].pools.labels.cloud-image (required)
Type: str

Refers to the name of an externally managed image in the cloud that already exists on the provider. The value of cloud-image should match the name of a previously configured entry from the cloud-images section of the provider. See providers.[aws].cloud-images. Mutually exclusive with providers.[aws].pools.labels.diskimage

providers.[aws].pools.labels.diskimage (required)
Type: str

Refers to provider’s diskimages, see providers.[aws].diskimages. Mutually exclusive with providers.[aws].pools.labels.cloud-image

providers.[aws].pools.labels.dedicated-host
Type: bool

If set to true, an AWS dedicated host will be allocated for the instance. Nodepool only supports running a single instance on dedicated hosts, so it will treat the host and the instance launched on it as a matched pair. The host will not be used for any other instances, and will be released when the associated Nodepool node is deleted.

If this option is set, the providers.[aws].pools.labels.use-spot option is not available, and providers.[aws].pools.availability-zone option is required.

providers.[aws].pools.labels.ebs-optimized
Default: False
Type: bool

Indicates whether EBS optimization (additional, dedicated throughput between Amazon EC2 and Amazon EBS,) has been enabled for the instance.

providers.[aws].pools.labels.instance-type
Type: str

Name of the flavor to use. Mutually exclusive with providers.[aws].pools.labels.fleet

providers.[aws].pools.labels.fleet
Type: dict

If specified, EC2 fleet API would be used for launching the instance. In this case, quota is not checked before launching the instance, but is taken into account after the instance is launched. Mutually exclusive with providers.[aws].pools.labels.instance-type

providers.[aws].pools.labels.fleet.instance-types
Type: list

Names of the flavors of the instance that to be launched.

providers.[aws].pools.labels.fleet.allocation-strategy (required)
Type: str

Allowed values for on On-Demand: lowest-price or prioritized. Allowed values for Spot: price-capacity-optimized, capacity-optimized, diversified or lowest-price

providers.[aws].pools.labels.iam-instance-profile
Type: dict

Used to attach an iam instance profile. Useful for giving access to services without needing any secrets.

providers.[aws].pools.labels.iam-instance-profile.name

Name of the instance profile. Mutually exclusive with providers.[aws].pools.labels.iam-instance-profile.arn

providers.[aws].pools.labels.iam-instance-profile.arn

ARN identifier of the profile. Mutually exclusive with providers.[aws].pools.labels.iam-instance-profile.name

providers.[aws].pools.labels.imdsv2
Type: str

Specify whether IMDSv2 is required. If this is omitted, then AWS defaults are used (usually equivalent to optional but may be influenced by the image used).

optional

Allows usage of IMDSv2 but do not require it. This sets the following metadata options:

HttpTokens is optional

HttpEndpoint is enabled

required

Require IMDSv2. This sets the following metadata options:

HttpTokens is required

HttpEndpoint is enabled

providers.[aws].pools.labels.key-name (required)
Type: string

The name of a keypair that will be used when booting each server.

providers.[aws].pools.labels.volume-type
Type: string

If given, the root EBS volume type

providers.[aws].pools.labels.volume-size
Type: int

If given, the size of the root EBS volume, in GiB.

providers.[aws].pools.labels.iops
Type: int

The number of I/O operations per second to be provisioned for the volume. The default varies based on the volume type; see the documentation under EBS volume type for the specific volume type for details.

providers.[aws].pools.labels.throughput
Type: int

The throughput of the volume in MiB/s. This is only valid for gp3 volumes.

providers.[aws].pools.labels.userdata
Default: None
Type: str

A string of userdata for a node. Example usage is to install cloud-init package on image which will apply the userdata. Additional info about options in cloud-config: https://cloudinit.readthedocs.io/en/latest/topics/examples.html

providers.[aws].pools.labels.tags
Default: None
Type: dict

A dictionary of tags to add to the EC2 instances. Values must be supplied as strings.
providers.[aws].pools.labels.dynamic-tags
Default: None
Type: dict
Similar to providers.[aws].pools.labels.tags, but is interpreted as a format string with the following values available:

request: Information about the request which prompted the creation of this node (note that the node may ultimately be used for a different request and in that case this information will not be updated).

id: The request ID.

labels: The list of labels in the request.

requestor: The name of the requestor.

requestor_data: Key/value information from the requestor.

relative_priority: The relative priority of the request.

event_id: The external event ID of the request.

created_time: The creation time of the request.

tenant_name: The name of the tenant associated with the request.

For example:
labels:
  - name: precise
    dynamic-tags:
      request_info: "Created for request {request.id}"
providers.[aws].pools.labels.use-spot
Default: False
Type: bool
When set to True, Nodepool will try to launch an Amazon EC2 Spot instance, instead of an On-Demand instance. Spot instances let you take advantage of unused EC2 capacity at a discount.

For example:
labels:
  - name: frugal
    use-spot: True
Note

As Amazon EC2 Spot instances take advantage of unused EC2 capacity, you may not get an instance, if demand is high. In addition, Amazon EC2 may interrupt your Spot instance and reclaim it with a two minutes warning upfront. Therefore, you might want to setup alternative nodesets as fallback.