* changes:
snapuserd: Add unit test for test merge code path
libsnapshot: Add vts_userspace_snapshot_test
libsnapshot: Integrate userspace snapshots APIs
fs-verity is required for new devices launched with R.
This allows files stored on /metadata to be protected by fsverity.
Bug: 199914227
Test: mini-keyctl padd asymmetric fsv-sepolicy .fs-verity \
< /system/etc/security/com.android.sepolicy.cert.der
cp /apex/com.android.sepolicy.apex/app/SEPolicy-33/SEPolicy-33.apk \
/metadata/sepolicy/
fsverity enable /metadata/sepolicy/SEPolicy-33.apk \
--signature=/apex/com.android.sepolicy.apex/etc/SEPolicy-33.apk.fsv_sig
Change-Id: I44434e3d026f1dbe6e261c365b3c70d3556a80b1
In previous kernels, overlayfs stored its xattrs with a "trusted."
prefix. This requires CAP_SYS_ADMIN. As a workaround, we carried
out-of-tree kernel patches to bypass the security checks on these attrs.
The 5.15 kernel however has a new mount option "userxattr". When this is
set, the "trusted." prefix is replaced with "user.", which eliminates
the CAP_SYS_ADMIN requirement.
On kernels >= 5.15 we can use this feature and drop some of our
out-of-tree patches.
Bug: 204981027
Test: adb remount on cuttlefish with >=5.15
Change-Id: I3f0ca637a62c949fe481eea84f2c682f1ff4517a
Most of the test cases are similar to dm-snapshot-merge.
Additional test cases have been added primarily to test
I/O's in parallel with merge.
Bug: 193863397
Test: snapuserd_test
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I2764e6971989c121be873cc425cac464f31ce85f
dm-user block device will be the snapshot device; thus, no
more explicit call to MapSnapshot(). Additionally, block device
name for dm-user will be the snapshot name so that mount works
seamlessly.
API's to query the snapshot status, merge progress has been
integrated. Since daemon requires base device for merge, we pass
additional parameter during initialization.
Add a new virtual a/b property flag to enable/disable
user-snapshots feature. Propagate this flag to init layer
for first stage mount during boot process.
Some minor cleanup and renaming of variables.
Bug: 193863443
Test: 1: Full OTA on CF and pixel and verify the merge completion.
Tested merge-resume path by rebooting device during merge.
2: Incremental OTA on CF and pixel
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I5088f40a55807946cd044b3987678ead3696d996
ReadFstabFromFp() have two callers right now, ReadFstabFromFile() and
ReadFstabFromDt(). ReadFstabFromFile() opens a FILE* and pass it to
ReadFstabFromFp(), and ReadFstabFromDt() wraps a std::string::c_str()
buffer in a FILE* adaptor with fmemopen().
There's no need for such adaptor, just change ReadFstabFromFp() to
accept std::string and we're good.
Bug: 206740783
Bug: 204056804
Test: atest CtsFsMgrTestCases
Test: m libfstab_fuzzer
Change-Id: I3f56a83ec5baf7b0d97a618a2c2bb6e31b67b5d9
android::base::Tokenize() is like android::base::Split() but ignores
empty tokens. Think strtok_r() and strsep().
C++-ify parsing code by replacing strtok_r() with Tokenize(), which
results in more concise and readable code.
Bug: 204056804
Test: atest CtsFsMgrTestCases
(cherry picked from commit 3c1b581fd5)
Change-Id: Icce8c1b5ad074421052f68fa138d90adb85cca27
ReadFstabFromFp() have two callers right now, ReadFstabFromFile() and
ReadFstabFromDt(). ReadFstabFromFile() opens a FILE* and pass it to
ReadFstabFromFp(), and ReadFstabFromDt() wraps a std::string::c_str()
buffer in a FILE* adaptor with fmemopen().
There's no need for such adaptor, just change ReadFstabFromFp() to
accept std::string and we're good.
Bug: 204056804
Test: atest CtsFsMgrTestCases
Test: m libfstab_fuzzer
Change-Id: I71190c7356a354aa06d1be3fdc4ca76611b3896d
Unfortunately, some recovery fstabs still specify the FDE options, where
they never really did anything anyway. Allow them for now, since it
seems preferable to restrict any breakage to the case that really
matters (devices actually configured to use FDE in their main fstab).
Bug: 191796797
Bug: 206025578
Change-Id: I85e35af8f42d2aef91f08816a67a71bbf756b211
There are some fstab files that specify "encryptable=footer" for
adoptable storage volumes, which contradicts the documentation which
says that it should be "encryptable=userdata". However, the argument
was previously being ignored anyway. To avoid unnecessarily breaking
such devices, ignore the argument to "encryptable".
Note that we continue to only allow "encryptable" in combination with
"voldmanaged". So, fstabs that use "encryptable" for FDE (rather than
for adoptable storage) should continue to be rejected.
Bug: 191796797
Change-Id: Idc4d5f9c01098f997e12be0022bea992439cec9c
android::base::Tokenize() is like android::base::Split() but ignores
empty tokens. Think strtok_r() and strsep().
C++-ify parsing code by replacing strtok_r() with Tokenize(), which
results in more concise and readable code.
Bug: 204056804
Test: atest CtsFsMgrTestCases
Change-Id: Ic1dd0eb97ef9ce6786dc2bf65cdee22a40d813e7
FDE is no longer supported, so there's no longer any need to ever
reserve a crypto footer.
Bug: 191796797
Change-Id: I79121188b0bcb7b00c16fda03b68b20c40c1e240
Since Android 10, new devices have been required to use FBE instead of
FDE. Therefore, the FDE code is no longer needed.
Bug: 191796797
Change-Id: I2f29ce5fa61c67325d6eb6cf6693787f8fa8a011
Since Android 10, new devices have been required to use FBE instead of
FDE. Therefore, the FDE code is no longer needed.
Make fs_mgr reject fstabs where FDE is enabled.
Unfortunately, there is a quirk where the "encryptable" flag (which was
originally meant just for FDE) was overloaded to identify adoptable
storage volumes. It appears that we have to keep supporting this use
case. Therefore, don't reject the "encryptable" flag completely.
Instead, just reject "encryptable" when it appears without
"voldmanaged", or without "userdata" as its argument.
Here are some references for how "encryptable=userdata" is being used to
identify adoptable storage volumes:
* https://source.android.com/devices/storage/config#adoptable_storage
* f26c7e9b12:system/vold/main.cpp;l=269
* f26c7e9b12:device/google/cuttlefish/shared/config/fstab.f2fs;l=17
* f26c7e9b12:device/generic/goldfish/fstab.ranchu;l=7
[ebiggers@: modified from a WIP CL by paulcrowley@]
Bug: 191796797
Change-Id: I3c4bbbe549cc6e24607f230fad27ea0d4d35ce09
This reverts commit 1cc9818642.
Reason for revert: Failing build - There is a dependency with https://r.android.com/c/1843772/3
Change-Id: Iccac56cf4c7012d8732b4338eb7055e1b2451e3b
Most of the test cases are similar to dm-snapshot-merge.
Additional test cases have been added primarily to test
I/O's in parallel with merge.
Bug: 193863397
Test: snapuserd_test
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: If7fede1be6d678c8665e1f9369fd3c69f5d3c4df
clean_scratch_files.rc seems to never work as intended due to
missing domain transition rules.
Add a 'remount' domain to the platform policies to allow
remount-related operations, including clean_scratch_files.
Merge remount and clean_scratch_files binary to simplify the added
policies.
Bug: 204836146
Test: 1. Use a VAB device, for example bramble-userdebug
2. adb remount system
3. adb reboot fastboot && fastboot flash system system.img
4. recovery (fastbootd) should mark the remount scratch as disabled
5. fastboot reboot && adb shell mount | grep scratch => nothing
6. adb logcat | grep 'clean_scratch_files|gsid' => shows that
disabled image "scratch" is removed during boot
Change-Id: I18d543868d1f37d43b483eae7517b707e46df1bd
Since we will be iterating forward for user-space
merge, we need to sort the blocks in increasing order
so that blocks can be batch merged if contiguous.
For dm-snapshot merging, we will continue to sort
in decreasing order.
Bug: 193863397
Test: Snapuserd_test
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I25fb5fce054f716a2ad0dddc0d0c3afef18bc7ad
Add new API to query the snapshot and merge status.
This will be used by libsnapshot.
Bug: 193863443
Test: Full OTA on CF
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I86cffff6a979e2e2bf1d8d1a1770e209eeb4a47d
Add new client API's for initiating and tracking merge.
These API's will be used by libsnapshot.
Track the merge completion in the server by walking through
all the partitions. Each worker thread will update the
merge completion as and when number of COW operations
are completed. Server will gather all the completions
of each partition and average it out. This is in sync
with the current merge completion tracking for dm-snapshot.
As a side effect, move the snapuserd_server.h/cpp files to
dm-snapshot-merge directory as it will only be a maintaining
code.
Bug: 193863443
Test: Snapuserd_test
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I031eb1a11b0f426aafbed3d39d85b0c22b9030fb
If the I/O request is for a sector which is not block
aligned, then we will not find the mapping directly
to a COW op as each op is block aligned. Thus, we
handle them by chopping the I/O request and processing
the unaligned data. Furthermore, if the request
doesn't map to any of the COW ops, then route the
I/O to the base device.
Bug: 196929997
Test: Full OTA on CF - during boot up, we get an I/O request
to read superblock on system partition which is not block aligned.
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ie40633c0c6bc0c87b681c051c74c0ac787e34d9c
Now that merging is done in user-space and
partitions are mounted off dm-user directly,
daemon will have to serve every I/O request.
Daemon has to handle this wherein we need to
check if the given I/O request block has been
modified in the OTA. Furthermore, if merge is
in-progress, we will have to synchronize with
the merge thread before servicing the I/O.
If the I/O request maps to a REPLACE or ZERO op,
we will just read the data from COW device.
If the I/O request maps to a COPY or XOR op,
the worker thread will have to synchronize
with the merge thread and if the merge is
in progress, fetch the data directly from RA buffer.
This patch handles I/O requests only if the
sectors are 4k aligned.
Bug: 196929997
Test: snapuserd_test
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I08562b8927e1c22dd9d9ef160e873280854eac99
Ensure we have at least one element before we try to access the last
element. Else the array index of the last element may underflow,
0ull - 1 == ~0ull == UINT64_MAX.
Bug: 204536075
Test: atest fiemap_writer_test
Change-Id: Ic390d108bf789cfe136fb5dfe2983f3c7d6f7e48
Bug: 201596822
Test: Local test (isolated to host)
$ adb -s 8BDAY00CM6 root
adbd is already running as root
$ adb -s 8BDAY00CM6 remount
Disabling verity for /system
Using overlayfs for /system
Disabling verity for /system_ext
Using overlayfs for /system_ext
Disabling verity for /vendor
Using overlayfs for /vendor
Disabling verity for /product
Using overlayfs for /product
remount succeeded.. now reboot device for settings refresh
$ echo $?
0
$ adb -s 8BDAY00CM6 unroot
restarting adbd as non root
$ adb -s 8BDAY00CM6 remount
Not running as root. Try "adb root" first.
remount failed
$ echo $?
3
Signed-off-by: Shaju Mathew <shaju@google.com>
Change-Id: I8e6d0735d2da0ed58cae867db2e4736abb86d351
The kernel can return different compatible ranges on each call, depending on
how it decides to merge contiguous extents in the results. To avoid the
complexity of requerying the ioctl, just do one query up to the maximum
allowed extent size.
Bug: 204536075
Test: install DSU on cuttlefish
fiemap_test
Change-Id: I4d569e3e6feed14c91a5f500296623888060dcad
This is probably an overkill, but still, add a collection of
interesting keywords that may appear in a fstab file.
Also add myself to the "CC" list of this fuzz target.
BYPASS_INCLUSIVE_LANGUAGE_REASON="slave" or MS_SLAVE is a mount flag
Bug: 204056804
Test: adb shell /data/fuzz/x86_64/libfstab_fuzzer/libfstab_fuzzer \
-dict=/data/fuzz/x86_64/libfstab_fuzzer/fstab.dict
Change-Id: Ife55837212a711dcbeed4f00d97da0cb022c7156
Just a modest initial implementation. Uses the fuzz data as the fstab
file content directly.
Bug: 204056804
Test: lunch aosp_cf_x86_64_phone-userdebug
SANITIZE_TARGET=address m libfstab_fuzzer
adb sync data
adb shell /data/fuzz/x86_64/libfstab_fuzzer/libfstab_fuzzer
Change-Id: I7976a6ee124e9b5da59cfa7f4bae9699be3f1474
Since there is no change in the buffer allocation
code with respect to snapshots in user-space, refactor
it so that we have a common code base.
Bug: 193863397
Test: Build
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I3a7b3d319f6d1532b1bb63af9884b64e1346bb50
Implement snapshot merge in the daemon. Following
are the important changes:
1: Spin up merge thread which does merging in user-space.
2: For ordered ops (COPY and XOR), read-ahead thread is used.
3: Read-ahead thread will read a fixed set of COW ops and cache
them in memory. Furthermore data is saved in scratch space buffer
in COW device.
4: No change in scratch space buffer - default 2MB buffer is allocated.
5: Merge thread and Read-ahead thread will work in lock step for merging
ordered ops.
6: Once the ordered ops are merged, REPLACE and XOR operations are
merged.
7: If there is a crash during merge, COW header tracks the number of
operations merged. No change in this code path.
8: Merge thread requires Base device for merging as opposed to using
the source device. Hence, while initializng the merge threads,
libsnapshot will have to pass the "Base" device to the daemon. This
is the same "Base" device which is passed to dm-snapshot during
snapshot creation.
Patch does not handle any communication with dm-user yet.
Bug: 193863397
Bug: 193863280
Bug: 193862712
Test: snapuserd_test on CF
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I14aab6eaa07ac68f2a3a23516ed9ba6567a35734
