nova Scheduling 配置

Nova中调度配置：

scheduler_driver_task_period = 60
scheduler_driver = nova.scheduler.filter_scheduler.FilterScheduler
scheduler_available_filters = nova.scheduler.filters.all_filters
scheduler_default_filters = RetryFilter, AvailabilityZoneFilter, 
RamFilter, DiskFilter, ComputeFilter, ComputeCapabilitiesFilter, 
ImagePropertiesFilter, ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter

scheduler Filtering

AggregateCoreFilter:
看组中host上的vcpu个数能否满足创建虚拟机的instancetype中的vcpu个数。通过 per-aggregate 的 cpu allocation_ratio 参数预定义的 CPU 核数进行过滤, 如果 per-aggregate的值没有被定义则回退到全局配置,如果宿主机在多个 aggregate 组中,且不止一个值被发现,则最小值会被使用

AggregateDiskFilter:
通过 per-aggregate 的 disk allocation ratio 参数预定义磁盘分配大小.方式过滤方式同上.

AggregateImagePropertiesIsolation:
匹配属性定义在 image的元数据中,用于对这些 aggregate 组进行匹配:

如果一个 host 属于一个 aggregate,这个 aggregate定义了一个或多个元数据并匹到一个 image 的属性,这个host 会作为通过该 image 启动实例的候选宿主机.
如果一个 host 不属于任何aggregate,则他可以从任何 image 启动虚拟机实例.

比如下面这个 aggregate 分组 myWinAgg 将 windows 系统作为元数据 os=windows

$ nova aggregate-details MyWinAgg
+----+----------+-------------------+------------+---------------+
| Id | Name     | Availability Zone | Hosts      | Metadata      |
+----+----------+-------------------+------------+---------------+
| 1  | MyWinAgg | None              | 'sf-devel' | 'os=windows'  |
+----+----------+-------------------+------------+---------------+

在这个理子中,因为下面 win-2012的镜像中有 windows 的属性, 通过他启动的虚拟机实例会再 sf-devel 分组上启动.

$ glance image-show Win-2012
+------------------+--------------------------------------+
| Property         | Value                                |
+------------------+--------------------------------------+
| Property 'os'    | windows                              |
| checksum         | f8a2eeee2dc65b3d9b6e63678955bd83     |
| container_format | ami                                  |
| created_at       | 2013-11-14T13:24:25                  |

You can configure the AggregateImagePropertiesIsolation filter by using the following options in the nova.conf file:
可以通过下面的方式配置AggregateImagePropertiesIsolation过滤

# Considers only keys matching the given namespace (string).
aggregate_image_properties_isolation_namespace = <None>

# Separator used between the namespace and keys (string).
aggregate_image_properties_isolation_separator = .

AggregateInstanceExtraSpecsFilter:
通过额外配置过滤,会匹配定义到设备组中的额外配置.将主机所属的aggregate的metadata属性，与创建虚拟机的instancetype中的extraspecs属性作比较。这个和ComputeCapabilitiesFilter有冲突,两者只能开一个同时，在extraspecs属性中，支持一些伪操作符（novaschedulerfiltersextraspecsops.py），比如extraspecs={'mem':'>= 512'}，那么host所属的aggregate的metadata={'mem':'1024'}时，就允许创建虚拟机。

ComputeCapabilitiesFilte

ComputeCapabilitiesFilter - checks that the capabilities provided by the host compute service satisfy any extra specifications associated with the instance type. It passes hosts that can create the specified instance type.

If an extra specs key contains a colon (:), anything before the colon is treated as a namespace and anything after the colon is treated as the key to be matched. If a namespace is present and is not capabilities, the filter ignores the namespace. For examplecapabilities:cpu_info:features is a valid scope format. For backward compatibility, the filter also treats the extra specs key as the key to be matched if no namespace is present; this action is highly discouraged because it conflicts with AggregateInstanceExtraSpecsFilter filter when you enable both filters

The extra specifications can have an operator at the beginning of the value string of a key/value pair. If there is no operator specified, then a default operator of s== is used. Valid operators are:
http://docs.openstack.org/developer/nova/filter_scheduler.html

AggregateIoOpsFilter
功能：根据设备组中的设备的IO负载过滤。 IO负载由host->numioops表示，与CONF.maxioopsperhost（默认是8）比较。io,maxioopsperhost.如果机器属于多个设备组,同样会按maxioopsperhost的最小值来应用

AggregateMultiTenancyIsolation
功能：租户和aggregate对应，实现创建虚拟机时租户独占资源。 host1所属的aggregate的metadata中filtertenantid=['tenant1', 'tenant2']，创建虚拟机的租户是tenant1，那么该虚拟机就允许创建在host1上。如果host1没有加入aggregate，或者aggregate没有配置metadata，则也允许创建虚拟机。

AggregateNumInstancesFilter
选择那些设备组中hoststate.numinstances < CONF.maxinstancesper_host的主机。

AggregateRamFilter
与 AggregateColeFilter 相同 ram allocation ratio=1.5

AggregateTypeAffinityFilter
aggregate->metadata-> instancetype，一个aggregate中只允许创建同一规格的虚拟机。instancetype

AggregateMultiTenancyIsolation

定义host只能创建某些租户的instance

ComputeFilter

host必须是operational and enabled，should always enable this filter

CoreFilter

要保证足够的CPU core,可以设置CPU超用

cpu_allocation_ratio = 16.0

NUMATopologyFilter

根据host的NUMA topology，通过定义instance的 flavorextra_specs和image properties来使用。

DifferentHostFilter

调度到different_host=a0cf03a5-d921-4877-bb5c-86d26cf818e1不同的host上。

$ nova boot --image cedef40a-ed67-4d10-800e-17455edce175 --flavor 1 
  --hint different_host=a0cf03a5-d921-4877-bb5c-86d26cf818e1 
  --hint different_host=8c19174f-4220-44f0-824a-cd1eeef10287 server-1

API

{
    "server": {
        "name": "server-1",
        "imageRef": "cedef40a-ed67-4d10-800e-17455edce175",
        "flavorRef": "1"
    },
    "os:scheduler_hints": {
        "different_host": [
            "a0cf03a5-d921-4877-bb5c-86d26cf818e1",
            "8c19174f-4220-44f0-824a-cd1eeef10287"
        ]
    }
}

DiskFilter

调度的hosts需要足够的剩余disk给 root and ephemeral.

$ nova hypervisor-stats
+----------------------+-------+
| Property             | Value |
+----------------------+-------+
| count                |  1    |
| current_workload     |  0    |
| disk_available_least |  29   |
| free_disk_gb         |  35   |
| free_ram_mb          |  3441 |
| local_gb             |  35   |
| local_gb_used        |  0    |
| memory_mb            |  3953 |
| memory_mb_used       |  512  |
| running_vms          |  0    |
| vcpus                |  2    |
| vcpus_used           |  0    |
+----------------------+-------+

可以看到disk_available_least要小于free_disk_gb, disk_available_least是计算virtual size而不是image的实际大小。如果image是sparse or copy on write，那么它

实际大小要比virtual size小。

调度依据disk_available_least而不是free_disk_gb。

disk_allocation_ratio = 1.0 默认不disk over commitment，如果需要，设置 > 1 ，同时注意 free_disk_gb。它代表的是真实的磁盘空余。

GroupAffinityFilter

已经被ServerGroupAffinityFilter取代了。需要设置scheduler hint，其中，group 为key

$ nova boot --image IMAGE_ID --flavor 1 --hint group=foo server-1

不能和 GroupAntiAffinityFilter同时使用，否则2个都不生效。

ImagePropertiesFilter

根据instance's image上定义的属性来过滤，熟悉包括

architecture, hypervisor type, hypervisor version (for Xen hypervisor type only), and virtual machine mode。

如果image->properties中没有这些属性，则通过过滤；如果有这些属性，而host->capabilities->supportedinstances没有，返回False。

比如，

$ glance image-update img-uuid --property architecture=arm --property hypervisor_type=qemu

IsolatedHostsFilter

定义特殊的image 集合只能运行在特殊的host集合上。设置flag restrict_isolated_hosts_to_isolated_images来enable。

需要事先在nova.conf里面设置isolated_images/isolated_hosts。

比如：

isolated_hosts = server1, server2
isolated_images = 342b492c-128f-4a42-8d3a-c5088cf27d13, ebd267a6-ca86-4d6c-9a0e-bd132d6b7d09

IoOpsFilter

host上的io并发超过max_io_ops_per_host 的会被过滤掉。

JsonFilter

自定义的json格式的scheduler hint。

支持的变量有：

$free_ram_mb
$free_disk_mb
$total_usable_ram_mb
$vcpus_total
$vcpus_used

比如

$ nova boot --image 827d564a-e636-4fc4-a376-d36f7ebe1747 
  --flavor 1 --hint query='[">=","$free_ram_mb",1024]' server1

在API中使用os:scheduler_hints key：

{
    "server": {
        "name": "server-1",
        "imageRef": "cedef40a-ed67-4d10-800e-17455edce175",
        "flavorRef": "1"
    },
    "os:scheduler_hints": {
        "query": "[&gt;=,$free_ram_mb,1024]"
    }
}

MetricsFilter

根据host上的meters weight_setting来过滤。

NumInstancesFilter

host上的instance数量大于max_instances_per_host则被过滤掉。

PciPassthroughFilter

Host上的设备满足flavor 里面定义的extra_specs 。

RamFilter

可以使用ram_allocation_ratio = 1.5来超用。

RetryFilter

如果调度的host，而host没能相应调度需求的次数超过scheduler_max_attempts，则过滤掉该host。

SameHostFilter

调度到和其他instance相同的host上。使用--hint flag。与DifferentHostFilter正好相反。

$ nova boot --image cedef40a-ed67-4d10-800e-17455edce175 --flavor 1 
  --hint same_host=a0cf03a5-d921-4877-bb5c-86d26cf818e1 
  --hint same_host=8c19174f-4220-44f0-824a-cd1eeef10287 server-1

With the API, use the os:scheduler_hints key:

{
    "server": {
        "name": "server-1",
        "imageRef": "cedef40a-ed67-4d10-800e-17455edce175",
        "flavorRef": "1"
    },
    "os:scheduler_hints": {
        "same_host": [
            "a0cf03a5-d921-4877-bb5c-86d26cf818e1",
            "8c19174f-4220-44f0-824a-cd1eeef10287"
        ]
    }
}

ServerGroupAffinityFilter

调度到某一组host上。先要创建带有affinity policy的server-group， instance创建是pass scheduler hint

$ nova server-group-create --policy affinity group-1
$ nova boot --image IMAGE_ID --flavor 1 --hint group=SERVER_GROUP_UUID server-1

ServerGroupAntiAffinityFilter

each instance in a group is on a different host

$ nova server-group-create --policy anti-affinity group-1
$ nova boot --image IMAGE_ID --flavor 1 --hint group=SERVER_GROUP_UUID server-1

SimpleCIDRAffinityFilter

基于host的IP subnet range来调度

$ nova boot --image cedef40a-ed67-4d10-800e-17455edce175 --flavor 1 
  --hint build_near_host_ip=192.168.1.1 --hint cidr=/24 server-1

TrustedFilter

host需要符合trust requirements

TypeAffinityFilter

根据host上的instance 类型来过滤，比如要求host上不能有不同类型的instance。

http://docs.openstack.org/liberty/config-reference/content/section_compute-scheduler.html