When allocating a partition with a size that is unaligned (to the
optimal alignment), the remaining sectors are wasted since they are
never reallocated. This is because the free list is guaranteed to only
contain optimally-aligned regions. Unfortunately this means when a
partition is resized, we are wasting a small amount of space each time.
On a non-A/B device, this could wind up being significant.
For example, with an alignment of 512KiB, a 4KiB partition at offset 0
will waste 508KiB of space. The next extent to be allocated by any
partition will start at the next 512KiB.
To address this, we check if the last extent for a partition can be
extended to cover the difference between its last sector and the next
optimally aligned sector. We also verify that this region was not
allocated to any other partition, and does not appear in the free list,
to make sure we're not stealing space that will be used somewhere else.
Bug: 120434950
Test: liblp_test gtest
Change-Id: I88689889d44a4d2c51e659241918aaf2c064e049
When this patch landed, it was thought that over-allocating partitions
had no undesired side effects. However there are cases where it is now
causing problems. When flashing a GSI for VTS, the vbmeta is appended to
the end of the partition. If libfs_avb cannot find this vbmeta by
looking at the end of the partition, verification will fail. Similar use
cases with verifying other dynamic partitions are expected to be
problematic, so for now we revert to block-size alignment.
This reverts commit dccfdca1e1.
Bug: 125113249
Test: liblp_test gtest
Change-Id: Ibfd66734b63265e9fbb2ccf6b670c168c36190bc
When flashing in fastbootd, we create a new MetadataBuilder using the
given super_empty.img and attempt to import the existing partition
table. This will fail if there is some incompatibility in the partition
layout or partition quotas.
This import code was accidentally double-accounting partitions when
determining if they could fit within the group quota, preventing
"fastboot flashall" once partitions reached a certain size.
Bug: 126930319
Test: liblp_test gtest
Change-Id: I89a69cba110b62719197c9a4885cfc5bcf8f009f
liblp treats the term "block size" ambiguously when it compares the logical
hardware block size with the file system block size (which for all
intents and purposes must be 4K). This warning is thus spurious on
devices with say a 512 logical block size. However, liblp's block size
should at least be a multiple of this, so change the check accordingly.
Bug: 123317012
Test: liblp_test gtest
Change-Id: I0f41f6bae60a512ab8d313e487c28606daa661a6
Bug: 122616553
Test: built and ran liblp_test on Pixel 3 with Q weekly build
Added missing __attribute__((packed)) in two metadata structures.
Fixed error logging message when repairing primary metadata.
Few very minor additions related to metadata validation.
Fixed an off by one error in the validation of partition name length.
Change-Id: Ic777baf97871c786a209da7c32bc13c1360a8482
Signed-off-by: Ramon Pantin <pantin@google.com>
When allocating a partition with a size that is unaligned (to the
optimal alignment), the remaining sectors are wasted since they are
never reallocated. This is because the free list is guaranteed to only
contain optimally-aligned regions. Unfortunately this means when a
partition is resized, we are wasting a small amount of space each time.
On a non-A/B device, this could wind up being significant.
For example, with an alignment of 512KiB, a 4KiB partition at offset 0
will waste 508KiB of space. The next extent to be allocated by any
partition will start at the next 512KiB.
To address this, we round up extents to the optimal alignment. This
means partitions may wind up slightly over-allocated, versus before,
where they would waste space by making it unavailable.
Bug: 120434950
Test: liblp_test gtest
Change-Id: I555209b301058555526cc4309f7049ae81cf877d
When allocating "b" partitions on a non-retrofit A/B device, prioritize
regions occuring in the second half of the super partition. To make this
effective, the region covering the midpoint sector is split into two
additional regions.
This will allow OTAs to avoid unecessary fragmentation, since each
slot's partitions will be grouped together, leaving a large chunk of
contiguous space available when the OTA deletes the target slot. Since
updates are not allowed to consume more than half of the super
partition, this should guarantee one extent per partition. Note that, if
this restriction is not in place (for example, a developer flashes a
massive "system_b"), then an additional extent will be allocated due to
the region that was split.
Bug: 120433288
Test: liblp_test gtest
Change-Id: I1797e59e14c8b0d4d0e6855a1d984e8159b21df2
This method was designed for a single-super model, and now needs to
change to accomodate two super partitions (system_a and system_b, for
retrofitting).
NewForUpdate is supposed to transition metadata from one block device
to the next for updates. For normal devices this is a no-op, since
metadata only exists on one partition (super). For retrofit devices,
metadata exists on system_a and system_b. This has two implications.
First, any references to the source slot must be rewritten. For example
"vendor_b" must become "vendor_a". However this is not true of partition
names. Partitions/extents are cleared in the updated metadata since they
no longer have any meaning (the block device list has been
rewritten). We also clear groups since they are re-added during OTA.
The reason we have to do this rewriting is that slot suffixes are
automatically applied in ReadMetadata. We do not have access to the
original unsuffixed metadata that was written by the initial OTA.
This was a conscious design decision, since it localizes retrofitting
idiosyncracies to just a few places (ReadMetadata, NewForUpdate, and
fastbootd), minimizing the number of external callers that have to
understand auto-slot-suffixing.
It would be arguably cleaner if retrofit metadata was always serialized
*without* slot suffixes, thereby making NewForUpdate a no-op. However
this would necessitate changes to the API elsewhere. The functions that
read partition names would have to take a slot suffix, and this would
further complicate MetadataBuilder and fastbootd. Another solution would
be to augment LpMetadata to retain unsuffixed information, but this is
probably not worthwhile given that retrofitting is intended to be
surgical, and will have a shorter lifespan than the non-retrofit case.
Bug: 116802789
Test: liblp_test gtest
Change-Id: I33596d92b38c47bc70bc0aa37ed04f6f0b9d4b6f
Retrofit devices will have two super partitions, spanning the A and B
slots separately. By design an OTA will never cause "A" or "B"
partitions to be assigned to the wrong super. However, the same is not
true of fastbootd, where it is possible to flash the inactive slot. We
do not want, for example, logical "system_a" flashing to super_b.
When interacting with partitions, fastbootd now extracts the slot suffix
from a GetSuperSlotSuffix() helper. On retrofit devices, if the partition
name has a slot, that slot will override FastbootDevice::GetCurrentSlot.
This forces partitions in the inactive slot to be assigned to the correct
super.
There are two consequences of this. First, partitions with no slot
suffix will default to the current slot. That means it is possible to
wind up with two "scratch" partitions, if "adb remount" is used on both
the "A" and "B" slots. However, only the active slot's "scratch" will be
visible to the user (either through adb or fastboot).
Second, if one slot does not have dynamic partitions, flashing will
default to fixed partitions. For example, if the A slot is logical and B
is not, flashing "system_a" will be logical and "system_b" will be
fixed. This works no matter which slot is active. We do not try to
upgrade the inactive slot to dynamic partitions.
Bug: 116802789
Test: fastboot set_active a
fastboot flashall # dynamic partitions
fastboot getvar is-logical:system_a # true
fastboot getvar is-logical:system_b # false
fastboot set_active b
fastboot flashall --skip-secondary
fastboot getvar is-logical:system_a # true
fastboot getvar is-logical:system_b # true
Booting both slots works.
Change-Id: Ib3c91944aaee1a96b2f5ad69c90e215bd6c5a2e8
This is needed for update_engine to properly clean old partitions on
retrofit devices.
Bug: 119687874
Test: liblp_test gtest
Change-Id: Ida9483ad3c127e357f45789540ebbedc9d3d3883
This adds a new MetadataBuilder constructor, NewForUpdate, that can be
used by update_engine to simplify upgrading metadata. It is safe to call
whether or not the device is a retrofit. If the metadata has block
devices assigned to a specific slot, and that slot matches the slot
suffix, it will ensure that an equivalent entry exists for the alternate
slot.
Thus, if the source slot is _a and the target slot is _b, and the
metadata has "system_a" as a block device but not "system_b", this will
automatically add "system_b" as a block device.
Bug: 116802789
Test: liblp_test gtest
Change-Id: Ie89d4dbf4c708b5705e658220227ebf33fcb1930
On retrofit devices, an OTA package or super_empty.img won't know which
slot it applies to. This is not an issue on devices shipping with
dynamic partitions, since they ship on the "a" slot.
To work around this, partitions and block devices can be flagged as
"auto-slot-suffixed". This indicates that ReadMetadata should
automatically append a slot suffix before returning the metadata. This
flag is added by MetadataBuilder when requested, and will be enabled via
lpmake separately.
After ReadMetadata has applied slot suffixes, it takes care to remove
the slot-suffix flag. This prevents the suffix from being applied twice,
if for example the metadata is then imported into a MetadataBuilder.
Bug: 116802789
Test: liblp_test gtest
retrofit device boots
Change-Id: Ic7de06d31253a8d5b8d15c0d936175ca2939f857
When updating the super partition, attempt to preserve partitions from
the other slot. If any partition can't be preserved, fail and require a
wipe (-w) to proceed. This allows two bootable builds to be flashed to
both slots.
The preserve operation can fail if the metadata is not compatible with
the old partition layout. For example, if the partition references a
group that no longer exists, or a group changed its capacity, or the
metadata's block device list or list contents changed.
Bug: N/A
Test: liblp_test gtest
fastboot flashall --skip-secondary
Change-Id: I53fdd29bc1f0ef132005a93d3cf1cdcd7f2fc05f
This patch allows the block device table in LpMetadataHeader to contain
additional partitions. MetadataBuilder can now resize partitions such
that are allocated across block devices as needed, however, it attempts
to minimize this by grouping free regions by device.
Bug: 116802789
Test: liblp_test gtests
device with super partition flashes and boots
Change-Id: I9cf74c8925faf154703eeba2a26546a152efcaa2
This change introduces an IPartitionOpener abstraction. Methods that
interact with live metadata, such as ReadMetadata, UpdatePartitionTable,
and FlashPartitionTable, now require an IPartitionOpener object. Its
purpose is dependency injection: it will make these methods much easier
to test when the super partition spans multiple block devices.
All non-test consumers should be using PartitionOpener, and as such,
some helper methods have been added that automatically create one.
Bug: 116802789
Test: liblp_test gtest
device with super partition boots
Change-Id: I76725a5830ef643c5007c152c00ccaad8085151f
This patch removes the alignment, block device size, and starting sector
fields from LpGeometry into a new LpMetadataBlockDevice struct. The
metadata now contains a table of these structs, and the table will have
exactly one entry representing the super partition.
This refactoring will make it easier to have logical partitions span
multiple physical partitions. When that happens, the table will be
allowed to have more than one entry, and the first entry of the table
will be considered the "root" of the super partition.
Bug: 116802789
Test: liblp_test gtest
device with logical partitions flashes and boots
Change-Id: I97f23beac0363182cb6ae78ba2595860950afcf0
These will help update_engine clear the target slot before applying an OTA.
Bug: 117182932
Test: liblp_test gtest
Change-Id: I6ad370e617f22f2098245a0028a93488f9ac8674
In preparation for supporting multiple block devices, this factors out
the free-list calculation for resizing partitions.
Additionally, it fixes a bug where space in between the first usable
sector and the first extent wasn't added to the free list.
Bug: 116802789
Test: liblp_test gtest
Change-Id: I965760eef0176020e9a5691ce1be2c8b5e0c8bc8
Align the first usable sector to the logical block size, if no other
alignment was specified. This fixes a bunch of warnings during certain
gtests (ones with unaligned metadata sizes). The warnings were coming
from MetadataBuilder::GrowPartition() which expects the first sector
to always be block-aligned.
Bug: 116802789
Test: liblp_test gtest
Change-Id: I8dcf502aa4c2ba0674c5b4dcb77a274f300ff0a3
Traditionally the first 512 bytes of a partition can be interpreted as
an MBR. To prevent any compatibility issues, we explicitly zero the
first 4096 bytes of the super partition (one logical block, on most
systems).
Bug: 116802789
Test: liblp_test gtest
device with super partition flashes and boots
Change-Id: I29688ca75dbb52442f1464e8ab35893678a4f79e
The group_indices variable was intended to facilitate this, but I forgot
to actually use it.
Bug: 116817738
Test: lpmake, lpdump super_empty.img
Change-Id: Ia0da50b56b6c09e277324ec9d7aea6ce48fdc10a
Now that backup metadata is stored at the start of the super partition,
this field is no longer needed. In actuality, it was not needed even
before then: both it and first_logical_sector exist for convenience,
since they can be re-derived at any time given an LpMetadataGeometry.
Bug: 116802789
Test: liblp_test gtest
device with dynamic partitions flashes and boots
Change-Id: I259a443097e689a0a9db7f822bbf1a52d40076dc
There's no reason to do this, since the fields are already in
LpMetadataGeometry. Removing this also simplifies multiple-block-device
support.
Bug: 116802789
Test: liblp_test gtest
Change-Id: Ib244a98fdd9d36c94a2dffd81bef68a1d5644ab9
Previously, metadata backups were stored at the end of the partition to
make them easy to locate. On older devices where the super partition
could span system/vendor partitions, we may want to leave the end of
each partition free to store an AVB footer. To allow this, we now store
geometry and metadata backups near the start of the partition instead.
They are still positioned at a fixed offset.
Bug: 116802789
Test: device boots after flashing new metadata
Change-Id: Ib173f251a4a13e290adcc4ac5cfbeb030eacda30
This patch introduces a new "groups" table in the super partition
metadata. Each entry denotes a named partition group with a maximum
size. All partitions now belong to a group, and the total size of
partitions in a group must not exceed its maximum size. This is enforced
by MetadataBuilder.
There is also a "default" group with no size restriction. This is used
for one-off partitions that aren't restricted by updates, for example,
the scratch partition for overlayfs, or partitions created through
fastbootd.
Bug: 116817738
Test: liblp gtest
Change-Id: I7049ffd35d326e41e25d01b1748cb53a584783a7
MetadataBuilder::AllocatableSpace() represents the total space
available. Adding MetadataBuilder::UsedSpace() to represent the
space used by the existing set of partitions. This will allow a
caller to predict if a partition can be resized (Grown), or to
create a partition to use up the free space (i.e. scratch).
Test: gTest liblp.UseAllDiskSpace
Bug: 109821005
Change-Id: Iae487d869fe18310c76b48c4281f1d6c1ee333cd
When adding extents to partitions, if the previous extent and new extent
are contiguous, merge them together to avoid allocating unnecessary
device-mapper targets.
Bug: 79173901
Test: liblp_test gtest
Change-Id: I80087df9aea8141c5e16f8d4cdb3dd7da02aee8c
This logs when partition tables update, when partitions resize, and when
partitions are unmapped from device mapper.
Bug: N/A
Test: N/A
Change-Id: I1125332c79fccc3ebc556b3b48856901e2503c47
DM_TABLE_LOAD will reject dm-linear entries if their size is not a
multiple of the backing device's logical block size. For example, a
partition of 10GiB+512 bytes will fail to map in device-mapper if the
logical block size is 4096 bytes. To address this, this patch adds a
few changes to liblp:
The block size given to lpmake is now recorded in LpGeometryMetadata.
The block size must be a multiple of the sector size. In addition,
partiton sizes are now aligned to the block size, and the super
partition must have enough free space to allocate at least one block (in
addition to storing metadata).
GrowPartition now has multiple checks that the block-size invariant is not
violated, to ensure that no invalid partition tables will be created.
Bug: 79173901
Test: liblp_test gtest
Change-Id: I484aac1f9b90ebd92dc1c89ce1e09cd89bbb441e
The partition resize algorithm duplicates a lot of logic because it
handles the final free interval separately from other free intervals.
This is unnecessary and makes it harder to change the actual algorithm.
This change makes GrowPartition() treat the final free space region the
same as free gaps in between partitions. It does this by converting the
extent list into a gap list, and then adds a final gap for the remainder
of the free space. The resize function no longer has to treat the end of
the disk separately.
This patch does not change the way partitions are allocated, it is
purely a refactoring.
Bug: 79173901
Test: liblp_test gtest
Change-Id: I4780f20b23fe021eac62de874b061857712c04fe
This change is to assist with implementing the fastbootd "resize-partition"
command. The GrowPartition and ShrinkPartition functions are now
private.
Bug: 78793464
Test: N/A
Change-Id: Ic66a3052359e2558663272ff6e014704206b197e
This makes offset calculations and library interactions much easier.
Bug: 79173901
Test: liblp_test gtest
Change-Id: I595c5435bd6bc166693a434ecdcd2d098185f963
This makes it easier to validate the device and make assumptions about
the device layout without having actual access.
Bug: 79173901
Test: liblp_test gtest
Change-Id: I53d24684020ec9210c8a17d155e738b3b2405806
This changes reader.h and writer.h to be private includes. A new liblp.h
header now contains the public API surface of those two files, as well
as some miscellanious functions previously in metadata_format.h.
Bug: 79173901
Test: N/A
Change-Id: I40c5dda0c8e5765f8bccfd5c17b4c800b41be77b
Block devices in the Linux kernel have a "minimum I/O request" size. The
minimum size is usually acquired by the block driver and can change
from device to device. When stacking devices (such as with
device-mapper), the kernel goes through great lengths to make sure this
alignment is respected for optimal I/O. In device-mapper's case,
misalignment can lead to kernel warnings and performance issues.
While this is unlikely to matter with a few targets, it could become
problematic on a large number of targets, and so we would prefer to
align all partition extents to the minimum I/O size.
We now support two new properties in the partition table geometry: an
"alignment", which is the minimum I/O size, and an "alignment offset",
which is an offset that when applied to sector 0, causes the sector to
be properly aligned within its parent device (for example, if a
physical partition is misaligned). All partition extents now begin on a
sector that respects this alignment.
One major caveat is that it is difficult for the initial partition table
to have the correct alignment without build system and/or flash tool
support. To accomodate this, all alignment is optional, and the lpmake
tool will support a default alignment of 1MiB as a failsafe.
Bug: 79173901
Test: liblp_test gtest
Change-Id: I5bc41b90aa085f4f30393951af0d2b37c4ac2a72
This library has methods to build, read, and modify logical partition tables
based on device-mapper targets. Targets currently supported are
dm-linear and dm-zero.
Note: this is a revert of a revert, with changes to fix Mac build
bustage.
Bug: 79173901
Test: N/A
Change-Id: If89a788d1919ce8ddc6eedaecc9687a92f111dfa
