o w7e @s UdZddlZddlZddlZddlmZddlmZmZddl m Z ddl m Z ddl mZmZddlmZdd lmZed Zed Zed Zed ZedZedZedZddiZeeZdZ ddde egeedgddgdZ!ee"d<ee!Zde#de de de$d df d!d"Z%d#d$Z&d%d&Z'dYd'd(Z(dZd*d+Z)d,d-Z*dZd.d/Z+d0d1Z,d2d3Z-    4 d[d5d6Z.d7d8Z/d9d:Z0d;d<Z1d=d>Z2d?d@Z3dAdBZ4dCdDZ5dEdFZ6dGdHZ7dIdJZ8dKdLZ9dMdNZ:dOdPZ;dQdRZdWdXZ?dS)\z1Disk Setup: Configure partitions and filesystems.N)dedent)subputil)Cloud)Config) MetaSchema get_meta_doc) ALL_DISTROS) PER_INSTANCEsfdisksgdisklsblkblkidblockdev partprobewipefsLANGCaiThis module is able to configure simple partition tables and filesystems. .. note:: for more detail about configuration options for disk setup, see the disk setup example .. note:: if a swap partition is being created via ``disk_setup`` then a ``fs_entry`` entry is also needed in order for mkswap to be run, otherwise when swap activation is later attempted it will fail. For convenience, aliases can be specified for disks using the ``device_aliases`` config key, which takes a dictionary of alias: path mappings. There are automatic aliases for ``swap`` and ``ephemeral``, where ``swap`` will always refer to the active swap partition and ``ephemeral`` will refer to the block device of the ephemeral image. Disk partitioning is done using the ``disk_setup`` directive. This config directive accepts a dictionary where each key is either a path to a block device or an alias specified in ``device_aliases``, and each value is the configuration options for the device. File system configuration is done using the ``fs_setup`` directive. This config directive accepts a list of filesystem configs. cc_disk_setupz Disk Setupz$Configure partitions and filesystemsan device_aliases: my_alias: /dev/sdb swap_disk: /dev/sdc disk_setup: my_alias: table_type: gpt layout: [50, 50] overwrite: true swap_disk: table_type: gpt layout: [[100, 82]] overwrite: true /dev/sdd: table_type: mbr layout: true overwrite: true fs_setup: - label: fs1 filesystem: ext4 device: my_alias.1 cmd: mkfs -t %(filesystem)s -L %(label)s %(device)s - label: fs2 device: my_alias.2 filesystem: ext4 - label: swap device: swap_disk.1 filesystem: swap - label: fs3 device: /dev/sdd1 filesystem: ext4 mounts: - ["my_alias.1", "/mnt1"] - ["my_alias.2", "/mnt2"] - ["swap_disk.1", "none", "swap", "sw", "0", "0"] - ["/dev/sdd1", "/mnt3"] disk_setupfs_setup)idnametitle descriptiondistros frequencyexamplesactivate_by_schema_keysmetarcfgcloudargsreturnc s|difdd}|d}t|trht||tdt||D]?\}}t|ts8td|q(ztdt j tjd|t ||fd Wq(t yg}zt td |WYd }~q(d }~ww|d } t| trtd t| t| || D]C}t|tstd|qztd|d} t j tjd| t|fd Wqt y}zt td|WYd }~qd }~wwd Sd S)z[ See doc/examples/cloud-config-disk-setup.txt for documentation on the format. device_aliasescs|}|p |p |SN)getdevice_name_to_device)candrr!r$@/usr/lib/python3/dist-packages/cloudinit/config/cc_disk_setup.pyalias_to_deviceys zhandle..alias_to_devicerzPartitioning disks: %szInvalid disk definition for %sz!Creating new partition table/diskzCreating partition on %s)logfuncmsgfuncr"z Failed partitioning operation %sNrzsetting up filesystems: %sz"Invalid file system definition: %szCreating new filesystem.devicezCreating fs for %sz%Failed during filesystem operation %s)r& isinstancedictupdate_disk_setup_devicesLOGdebugstritemswarningrlog_timemkpart Exceptionlogexclistupdate_fs_setup_devicesmkfs) rr r!r"r,rdisk definitionerr0r*r)r+handlersZ                rCcCst|D]:}||}|dus||krq||vr!td||||=||||<t||tr4|||d<||=td||qdS)Nz0Replacing %s in disk_setup for translation of %s _orignamez,updated disk_setup device entry '%s' to '%s')r=r4infor1r2r5)rtformerorigname transformedr*r*r+r3s*   r3cCs|D]Q}t|tstd|q|d}|durqt|\}}||}|dur;|}td|||||d<||d<|rSd|vrOtd||||d|d<||d<qdS)Nz"entry in disk_setup not a dict: %sr0z%s is mapped to disk=%s part=%srD partitionzKPartition '%s' from dotted device name '%s' overrides 'partition' key in %s _partition)r1r2r4r8r&rexpand_dotted_devnamer5)rrFrArGdevparttformedr*r*r+r>s8    r>ccsBt|}|r||d}dd|DD] \}}||fVqdS)zd Returns the key/value pairs of output sent as string like: FOO='BAR' HOME='127.0.0.1' NcSsg|]}|dqS)=)split.0xr*r*r+ sz"value_splitter..)shlexrP)valuesstart_valueskeyvaluer*r*r+value_splitters   r[Fc cstddd|g}|r|dd}z t|\}}Wnty/}z td||f|d}~wwdd|D}|D]}ddddd }t|D] \} } | || <qJ|Vq=dS) a Enumerate the elements of a child device. Parameters: device: the kernel device name nodeps : don't enumerate children devices Return a dict describing the disk: type: the entry type, i.e disk or part fstype: the filesystem type, if it exists label: file system label, if it exists name: the device name, i.e. sda z--pairsz--outputzNAME,TYPE,FSTYPE,LABELz--nodepsN"Failed during disk check for %s %scSs g|] }t|dkr|qS)r)lenrPrQr*r*r+rTs z"enumerate_disk..)rtypefstypelabel) LSBLK_CMDappendrr; RuntimeErrorstrip splitlinesr[lower) r0nodeps lsblk_cmdrE_errrBpartsrMdrYrZr*r*r+enumerate_disks>  rlcCs.t|ddD]}d|vr|dSqdS)z@ Return the device type of the device by calling lsblk. T)rgr^N)rlrf)r0rkr*r*r+ device_type&s rmcCsVd}zt|}Wntytd|YdSw|r!|dkr!dS|s)|dkr)dSdS)z0 Check if the device is a valid device. zQuery against device %s failedFrMTr@)rmr;r4r8)rrId_typer*r*r+is_device_valid1s     rpc Csd\}}}}tdd|g}z tj|ddgd\}}Wnty.}z td||f|d}~ww|r^t|d kr^t|d d D]\}} |d krL| }q?|d krU| }q?|d kr]| }q?|||fS)z Check if the device has a filesystem on it Output of blkid is generally something like: /dev/sda: LABEL="Backup500G" UUID="..." TYPE="ext4" Return values are device, label, type, uuid )NNNNz-cz /dev/nullr)rcsr\N)rWr`r^uuid) BLKID_CMDrr;rcr]rer[rf) r0outr`fs_typert blkid_cmdrirBrYrZr*r*r+check_fsCs.      rycCst|\}}}|S)z7 Returns true if the device has a file system. )ry)r0_rwr*r*r+ is_filesystemcsr{TcCs|durd}|s ddg}d}t|D]W}|d|kr(|dur(d|ddfS|d|krB|r6|d |ks8|sBd|dd fS|d |vri|d dksR|drTd }|d dkr[q|dsid|ddfSq|sp|dfStd d S)a Find a device that is either matches the spec, or the first The return is value is (, ) where the device is the device to use and the bool is whether the device matches the fs_type and label. Note: This works with GPT partition tables! Nrnr@rMFr_/dev/%srr`Tr^z5Failed to find device during available device search.)NF)rlr4r8)r0rwr` valid_targets label_match replace_fsraw_device_usedrkr*r*r+find_device_nodeks2     rcCs2ttt|dkr dSt|\}}}|rdSdS)z Check if the device is currently used. Returns true if the device has either a file system or a partition entry is no filesystem found on the disk. rsTF)r]r=rlry)r0rz check_fstyper*r*r+ is_disk_useds  rc Cshzttd|g\}}ttd|g\}}Wnty+}z td||f|d}~wwt|t|S)Nz --getsize64z--getsszFailed to get %s size %s)r BLKDEV_CMDr;rcint)r0 size_in_bytesrz sector_sizerBr*r*r+ get_hdd_sizesrc Cst|td|g}z tj|d|d\}}Wnty+}z td||f|d}~wwg}|D]C}|}t|dkr?q2||dvru|ddvrNq2d} t t d t|d d D]} ||  ro|| d kro|| } nq[| | q2|S) z Returns true if the partition layout matches the one on the disk Layout should be a list of values. At this time, this only verifies that the number of partitions and their labels is correct. z-l%s data(Error running partition command on %s %sNr)extendedemptyrsT)reverse/) read_parttbl SFDISK_CMDrr;rcrerPr]rfsortedrangeisdigitrb) r0layoutprt_cmdrvrirB found_layoutline_line type_labelrSr*r*r+check_partition_mbr_layouts:      rc Cstd|g}z tj|td\}}Wnty%}z td||f|d}~wwt|}|D] }|dr9nq.dd|D}g} |D]} t | dkrX| drX| d d } | | qE| S) Nz-p) update_envrNumbercSsg|] }|dqS))rdrP)rRrr*r*r+rTsz.check_partition_gpt_layout..00rrq) SGDISK_CMDr LANG_C_ENVr;rciterrerd startswithr]endswithrb) r0rrrvrirB out_linesrcodescleanedcoder*r*r+check_partition_gpt_layouts.     rcCsd|kr t||}nd|krt||}ntdtd||||t|tr2|r0t|dkr0dSdSt|t|krcdd |D}td ||t||D]\}}|d ur`t |t |kr`dSqMdSdS) z See if the partition lay out matches. This is future a future proofing function. In order to add support for other disk layout schemes, add a function called check_partition_%s_layout gptmbrUnable to determine table typez6called check_partition_%s_layout(%s, %s), returned: %srsTFcSs*g|]}t|ttfrt|dndqS)rsN)r1tupler=r6rQr*r*r+rT1sz*check_partition_layout..zLayout types=%s. Found types=%sN) rrrcr4r5r1boolr]zipr6) table_typer0rr layout_typesitypeftyper*r*r+check_partition_layouts:   rc Cs t|ts t|tr dSt|dkrt|tst|ts tdt|}|dkr,tdg}d}|D]@}d}|}|d7}t|trQt|dkrMtd ||\}}tt|t|d }||kri|d |q2|d ||fq2d |} t|dkrtd| | S)a@ Calculate the layout of the partition table. Partition sizes are defined as percentage values or a tuple of percentage and partition type. For example: [ 33, [66: 82] ] Defines the first partition to be a size of 1/3 the disk, while the remaining 2/3's will be of type Linux Swap. z,,83rzPartition layout is invalidrz$Only simply partitioning is allowed.Srsrq%Partition was incorrectly defined: %sdz,,%sz,%s,%s z-Calculated partition definition is too big %s) r1r=rr]rcrfloatrbjoin) sizer last_part_numpart_definitionpart_numrM part_typepercent part_sizesfdisk_definitionr*r*r+get_partition_mbr_layout?sB     rcCst|tr dddgfgSg}|D]4}t|tr(t|dkr#td||\}}n|}d}tt|t|d}||dd|gfqd|ddd<|S)Nrrqrrz+{}r) r1rr=r]rcrrrbformat)rrpartition_specsrIrpartition_typerr*r*r+get_partition_gpt_layoutxs    rcCsvd}d}d}t|d"}||||| tj||||Wdn1s0wYt|dS)Nizrb+)openwriteseekosSEEK_ENDflushr)r0null start_lenend_lenfpr*r*r+purge_disk_ptables   rc Cst|D]6}|ddvr:tdd|dg}ztd|dt|Wqty9}z td|d|d}~wwqt|dS) z' Remove parition table entries r^)r@cryptz--allr|rzPurging filesystem on /dev/%szFailed FS purge of /dev/%sN)rl WIPEFS_CMDr4rErr;rcr)r0rk wipefs_cmdrBr*r*r+ purge_disks    rcCs,d|kr t||Sd|krt||Std)z Call the appropriate function for creating the table definition. Returns the table definition This is a future proofing function. To add support for other layouts, simply add a "get_partition_%s_layout" function. rrr)rrrc)rrrr*r*r+get_partition_layouts   rc Csttdur t|g}ntd|g}tzt|Wnty3}zttd|WYd}~nd}~wwtdS)zq `Partprobe` is preferred over `blkdev` since it is more reliably able to probe the partition table. Nz --rereadptz%Failed reading the partition table %s) PARTPROBE_CMDrrudevadm_settlerr;r<r4)r0 probe_cmdrBr*r*r+rs   rc CsXtd|g}z tj|d|dWnty%}z td||f|d}~wwt|dS)zV Break out of mbr partition to allow for future partition types, i.e. gpt z--forcerrz Failed to partition device %s %sN)rrr;rcr)r0rrrBr*r*r+exec_mkpart_mbrs   rc CszBttd|gt|D]3\}\}\}}|d7}ttdd||||g|dur@t|dd}ttdd|||gq WntyPtd |wt |dS) Nz-Zrsz-nz{}:{}:{}r0z-tz{}:{}zFailed to partition device %s) rr enumeraterr6ljustr;r4r8r)r0rindexrrWendpinputr*r*r+exec_mkpart_gpts.    rcCs8tj|sttj|std|tdS)z?Assert that device exists and settle so it is fully recognized.zBDevice %s did not exist and was not created with a udevadm settle.N)rpathexistsrrrcr0r*r*r+assert_and_settle_devices   rcCst|tj|}td||dd}|dd}|dd}tdt|tr.|r0|s7tdd Std |t |sIt d j |d t|t r_| d kr_tdt|d Stdt|||rqtddStd|st|st|rtd|d Std|t|}tdt|||}td|td|d|krt||nd|krt||nt dtd|d S)a Creates the partition table. Parameters: definition: dictionary describing how to create the partition. The following are supported values in the dict: overwrite: Should the partition table be created regardless of any pre-exisiting data? layout: the layout of the partition table table_type: Which partition table to use, defaults to MBR device: the device to work on. z!Checking values for %s definition overwriteFrrrz Checking against default devicesz)Device is not to be partitioned, skippingNz'Checking if device %s is a valid devicez%Device {device} is not a disk device!rremovez,Instructed to remove partition table entriesz!Checking if device layout matchesz"Device partitioning layout matchesTz'Checking if device is safe to partitionz-Skipping partitioning on configured device %szChecking for device size of %szCalculating partition layoutz Layout is: %szCreating partition table on %srrzPartition table created for %s)rrrrealpathr4r5r&r1rrprcrr6rfrrrr{rrrr)r0rArrr device_sizerr*r*r+r:sN                      r:cCsDdddddd}d|vrd}||vr||Std|dS)z9 A force flag might be -F or -F, this look it up z-Fz-f)extbtrfsxfsreiserfsswaprzForce flag for %s is unknown.rn)rfr4r8)fsflagsr*r*r+lookup_force_flagXs   rc Csr|d}|d}t|dd}|d}|dg}|dg}|dd }|d d }t|tj|}td ||rE|r|rVd ||f}td ||td|t |\} } } td|| | | |kr| |krtd||std|dSt d|nmtd|nf|rt| dvr|} td|d} | dkrd } t |||| |d\}}td| ||rtddS|s|r|rtd||std|dSn|rt| dkrtd|ntddStd ||||s t d!|dS|s|st d"j|d#d }|r<|d|||d$}d}|r2t d%||r;t d&|nMtd'|}|sMtd(|}|sYt d)||dS|g}|rf|d*|g|spt|d+kr|t|}|r||||r||||td,||td-t|z tj||d.WdSty}z t d/||f|d}~ww)0a Create a file system on the device. label: defines the label to use on the device fs_cfg: defines how the filesystem is to look The following values are required generally: device: which device or cloud defined default_device filesystem: which file system type overwrite: indiscriminately create the file system partition: when device does not define a partition, setting this to a number will mean device + partition. When set to 'auto', the first free device or the first device which matches both label and type will be used. 'any' means the first filesystem that matches on the device. When 'cmd' is provided then no other parameter is required. r`r0rIany filesystemcmd extra_optsrFrz#Checking %s against default devicesz%s%sz%Manual request of partition %s for %szChecking device %sz0Device '%s' has check_label='%s' check_fstype=%sz Existing file system found at %sz"Device %s has required file systemNzDestroying filesystem on %sz"Device %s is cleared for formating)autorz,Identifying device to create %s filesytem onT)rwr`r~rz(Automatic device for %s identified as %sz+Found filesystem match, skipping formating.z*Replacing file system on %s as instructed.zENo device available that matches request. Skipping fs creation for %snonez.Using the raw device to place filesystem %s onz(Error in device identification handling.z;File system type '%s' with label '%s' will be created on %szDevice is not known: %szENo way to create filesystem '{label}'. fs_type or fs_cmd must be set.)r`)r`rr0z8fs_setup:overwrite ignored because cmd was specified: %sz9fs_setup:extra_opts ignored because cmd was specified: %szmkfs.%szmk%sz.Cannot create fstype '%s'. No mkfs.%s commandz-Lr@zCreating file system %s on %sz Using cmd: %s)shellzFailed to exec of '%s': %s)r&r6rrrrr4r5rryr8rfrrcrrwhichextendrmrrbr;)fs_cfgr`r0rIrwfs_cmdfs_opts fs_replacer check_labelrrzodevicer~reusermkfs_cmd force_flagrBr*r*r+r?ns                          r?r%)F)NNNTN)@__doc__loggingrrUtextwrapr cloudinitrrcloudinit.cloudrcloudinit.configrcloudinit.config.schemarrcloudinit.distrosr cloudinit.settingsr rrrrarurrrr getLogger__name__r4MODULE_DESCRIPTIONr__annotations__r6r=rCr3r>r[rlrmrpryr{rrrrrrrrrrrrrrrr:rr?r*r*r*r+s             )35 & 2    8 ').9G