[zz]libvirt中CPU和内存的细粒度管理机制

内存

struct _virDomainDef {

...

   struct {
        unsigned long max_balloon;
        unsigned long cur_balloon;

        unsigned long hugepage_backed;
        unsigned long hard_limit;
        unsigned long soft_limit;
        unsigned long min_guarantee;
        unsigned long swap_hard_limit;
    } mem;

...

}

选项：

1. hard_limit： ---> memory.limit_in_bytes

it represents the maximum memory the guest can use.

guest能使用的最大内存

2. soft_limit ---> memory.soft_limit_in_bytes

it represents the memory upper limit enforced during memory contention.

内存紧张时的保证。

比如：1G内存的物理主机，我们分配两个虚拟机器，设置hard_limit均为600M。但这样如果在资源紧张时就傻逼了。

需要设置soft_limit=500M，保证最多只有这么多内存可被用。

3. min_guarantee --->

it represents the minimum memory guaranteed to be reserved for the guest

注：lxc未设置这个

4. swap_hard_limit ---> memory.memsw.limit_in_bytes

it represents the maximum swap plus memory the guest can use. This limit has to be more than

接口：

virDomainSetMemoryParameters：

virDomainGetMemoryParameters

lxc实现： lxcDomainSetMemoryParameters lxcDomainGetMemoryParameters{

根本不同的参数，设置或者获取hard_limit, soft_limit, swap_hard_limit 的值

}

virDomainSetMemory:(不停用虚拟机的情况下动态增加内存的使用值)

Dynamically change the target amount of physical memory allocated to a domain.

在lxc的实现中，就是设置：memory.limit_in_bytes的值，跟hard_limit一致

virDomainSetMaxMemory:

Dynamically change the maximum amount of physical memory allocated to a domain

在lxc的实现中lxcDomainSetMaxMemory{ // 这个地方跟接口语义不太符合

   if (newmax < vm->def->mem.cur_balloon) {
        lxcError(VIR_ERR_INVALID_ARG,
                         "%s", _("Cannot set max memory lower than current memory"));
        goto cleanup;
    }

    vm->def->mem.max_balloon = newmax; // 这个值没有回写到cgroup中，而是用来在lxcDomainSetMemory中作做判断用

}

xml中的配置选项：

<memory>286954</memory> 这个参数必须有，不然启动不成功

  <memtune>
    <hard_limit>1048576</hard_limit>
    <soft_limit>131072</soft_limit>
    <swap_hard_limit>2097152</swap_hard_limit>
    <min_guarantee>65536</min_guarantee> #对lx无效
</memtune>

lxc相关：

我们直接用virDomainSetMemoryParameters来控制参数， virDomainSetMemory之类不好用。

总结：lxc中需要配置的参数：

hard_limit，soft_limit，swap_hard_limit

可以通过virDomainSetMemory动态增加内存，但增加不能超过max_balloon的值，所以，当增加的内存指超过max_balloon 的时候，需要先调用virDomainSetMaxMemory增加内存，然后再调用virDomainSetMemory

Python接口：

CPU

CGROUP主要设置参数

cpu.shares

cpuset.cpus

选项：

struct _virDomainDef {

unsigned short vcpus;

    unsigned short maxvcpus;
    int cpumasklen;
    char *cpumask;

    struct {
        unsigned long shares;
        int nvcpupin;
        virDomainVcpupinDefPtr *vcpupin;
    } cputune;

}

相关接口：

_virSchedParameter

virDomainGetSchedulerParameters

virDomainSetSchedulerParameters

Python接口：

/**
* virDomainGetMaxVcpus:
* @domain: pointer to domain object
*
* Provides the maximum number of virtual CPUs supported for
* the guest VM. If the guest is inactive, this is basically
* the same as virConnectGetMaxVcpus(). If the guest is running
* this will reflect the maximum number of virtual CPUs the
* guest was booted with. For more details, see virDomainGetVcpusFlags().
*
* Returns the maximum of virtual CPU or -1 in case of error.
*/

virDomainGetMaxVcpus()：

/**
* virDomainSetVcpus:
* @domain: pointer to domain object, or NULL for Domain0
* @nvcpus: the new number of virtual CPUs for this domain
*
* Dynamically change the number of virtual CPUs used by the domain.
* Note that this call may fail if the underlying virtualization hypervisor
* does not support it or if growing the number is arbitrary limited.
* This function requires privileged access to the hypervisor.
*
* This command only changes the runtime configuration of the domain,
* so can only be called on an active domain. It is hypervisor-dependent
* whether it also affects persistent configuration; for more control,
* use virDomainSetVcpusFlags().
*
* Returns 0 in case of success, -1 in case of failure.
*/

virDomainSetVcpus()

virDomainGetVcpusFlags/virDomainSetVcpusFlags()

/**
* virDomainPinVcpu:
* @domain: pointer to domain object, or NULL for Domain0
* @vcpu: virtual CPU number
* @cpumap: pointer to a bit map of real CPUs (in 8-bit bytes) (IN)
*      Each bit set to 1 means that corresponding CPU is usable.
*      Bytes are stored in little-endian order: CPU0-7, 8-15...
*      In each byte, lowest CPU number is least significant bit.
* @maplen: number of bytes in cpumap, from 1 up to size of CPU map in
*     underlying virtualization system (Xen...).
*     If maplen < size, missing bytes are set to zero.
*     If maplen > size, failure code is returned.
*
* Dynamically change the real CPUs which can be allocated to a virtual CPU.
* This function requires privileged access to the hypervisor.
*
* This command only changes the runtime configuration of the domain,
* so can only be called on an active domain.
*
* Returns 0 in case of success, -1 in case of failure.
*/

virDomainPinVcpu()

以上接口lxc均为实现

virDomainGetSchedulerType()

lxc实现：lxcGetSchedulerType{

schedulerType = strdup("posix");

return schedulerType; // 简单地返回,不知道有什么鸟用

}

virDomainSetSchedulerParameters()

lxc实现：

virDomainGetSchedulerParameters()

virCgroupCpuSetInherit

xml中的配置选项：

/*应该是shares，而不是cpushares (virXPathULong("string(./cputune/shares[1])", ctxt, 文档中这个地方时有错误的*/
</cputune>