adb_debug.prop is migrated too. And ramdisk_available is added to all
dependencies.
Bug: 187196593
Test: boot
Change-Id: I59cd149e0021211b8fd59c44b93bbf18dc8637bf
Merged-In: I59cd149e0021211b8fd59c44b93bbf18dc8637bf
This refactoring moves ImageManager creation out of SnapshotManager,
where it had always been rather awkward, and into IDeviceInfo so it can
participate in dependency injection.
The "first-stage init" state is now part of IDeviceInfo as well.
Bug: 187396878
Test: vts_libsnapshot_test
libsnapshot_fuzzer_test
Change-Id: Ic4b3e93527cc074ec18c0c26440dbdb2504d0d5e
These are unit tests for important VAB functionality. They are very
quick to run and should be included in presubmit testing.
Bug: N/A
Test: th green
Change-Id: I02e24c76df365e9fb2d68f904e930dce60b9bdaf
Introduce read-ahead mechanism for COW copy ops.
1: Read-ahead thread will read from base device
and store the data in scratch space along with the metadata.
2: Worker threads during merge will retrieve the data
from read-ahead cache
3: Fixed set of blocks are read during each cycle by the read-ahead
thread.
4: When the last block in the region is merged, read-ahead thread
makes forward progress.
Scratch space is set to 2MB and is only used from COW copy operations.
We can extend this to Replace Ops based on performance evaluation.
Performance:
As mentioned in bug 181883791, Incremental OTA of size 55M with
235K copy operations where every block is moved by 4k:
Without read-ahead: 40 Minutes for merge completion
With read-ahead: 21 Minutes for merge completion
Bug: 183863613
Test: 1: Full OTA - no regression observed.
2: Incremental OTA - with older COW format. Daemon will just skip
the read-ahead feature for older COW format.
3: Incremental OTA - with new COW format.
4: Reboot and crash kernel when multiple times when incremental OTA is in-flight.
Verify post reboot, read-ahead thread re-constructs the data from scratch
space.
5: No regression observed in RSS-Anon memory usage when merge in-flight.
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ic565bfbee3e9fcfc94af694596dbf44c0877639f
Example log line:
update_engine: Block device was lazily unmounted and is still in-use:
/dev/block/dm-28; possibly open file descriptor or attached loop device.
This will help diagnose bugs such as b/184715543 in the future.
Bug: N/A
Test: manual test
Change-Id: Ia6b17fe9bd1796d59be7fc0b355218509acfd4af
Add worker threads per partition to serve the IO request.
Remove memset of buffer in IO path which was impacting
4k IO performance.
update_verifier performance:
1: ~10-12 seconds with this change (both on full OTA and incremental
OTA); ~70 seconds observed without this changeset
2: ~8 seconds without the daemon once merge is completed
and snapshot devices are removed.
Bug: 181293939
Test: update_verifier, full OTA, incremental OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Id90887f3f4a664ee5d39433715d1c166acbd6c60
Allow batch merge of copy operations during merge.
When metadata is read from COW device, assign
the chunk-id by validating there is no overlap
of copy operations. Furthermore, detect the blocks
which are contiguous and batch merge them.
No regression in merge time for full OTA (~35-40 seconds)
Merge time for incremental OTA of ~200M takes about 2 minutes
as compared to 15-20+ minutes without this change.
Add unit test to test ReadMetadata() functionality.
Multiple incremental OTA and full OTA test done on pixel.
adb reboot during merge and validate the merge resume operations.
Bug: 179629624
Test: incremental OTA and full OTA on pixel,
cow_snapuserd_test
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I4cd84e4923e42afacc796b8cec01738b1bb1f420
This is in preparation for expanding the command-line features of
snapuserd.
Bug: N/A
Test: builds
Change-Id: Id33c4f190dc0f99cd436f0e9a6b1d6ee92e245e4
... as being able to use cc_library in header_libs is not
intended.
Bug: 173252016
Test: m checkbuild
Change-Id: I624ee84728c59744c89ab51db567900c8eaf0571
This reverts commit 68ba6695bd.
Bug: 174587489
Test: build and manual inspect
Reason for revert: revert workaround. Fix is in I71227d807244b.
Change-Id: I71b1c0f933eeba9dd8d8fbef3a682657f005cc3b
This is happening accidentally because of a bug in Soong. In the
meantime, add a separate snapuserd.vendor_ramdisk that does not have an
init_rc section.
Bug: N/A
Test: manual test
Change-Id: I9dcafb681f209932fa13c5c55478f0e486e53306
Refact cow_snapuserd unit test. This will
test code changes in daemon in isolation. Currently it tests
1: Reading snapshot device and validate data
2: Merge and validate data.
More tests related to merge will be added later; but this
should provide a baseline unit test for daemon.
BUG: 168311203
Test: cow_snapuserd_test
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I4d77c435a9f045204a4ff3ece524c857f9fba50f
When Virtual A/B Compression is enabled, the manifest contains the
predicted COW size. Use this instead of the algorithm based on the
kernel COW format.
Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: I545679b4834957ff80a930d91cb44afbadebb66c
This patch introduces the fundamentals needed to support booting off
dm-user. First, a method has been added to start snapuserd in
first-stage init. It simply forks and execs, creates a specially named
first-stage socket, then waits for requests.
Next, a new method has been added to SnapshotManager to perform a
second-stage handoff. This works by first launching a second copy of
snapuserd using init's normal service management functionality. The new
snapuserd runs alongside the original, but has correct privileges and a
correct selinux context. Next, we inspect each COW device, and if its
table uses dm-user, we replace the table with a renamed control
device. The new control device is bound to the new snapuserd.
device-mapper guarantees that such a table swap is safe. It flushes I/O
to the old table and then replaces it with the new table. Once the new
table is in place, the old dm-user control devices are automatically
destroyed. Thus, once all dm-user devices has been transitioned, the
first-stage daemon is idle and can gracefully exit.
This patch does not modify init. A few changes will be needed on top of
this patch:
(1) CreateLogicalAndSnapshotPartitions will need further changes to
start the first-stage daemon and track its pid. Additionally, it will
need to ensure the named socket file is deleted, so there is no further
IPC allowed after partitions are completed.
(2) init will need to propagate the pid to second-stage init so the
process can be killed (or signalled).
(3) first-stage snapuserd will need to gracefully exit once it has no
active handler threads.
(4) second-stage init will need to invoke the transition helper on
SnapshotMaanager, ideally as soon as feasible.
Bug: 168259959
Test: manual test
Change-Id: I54dec2edf85ed95f11ab4518eb3d7dbaf0bdcbfd
* changes:
libsnapshot: Remove the timeout on client recv().
libsnapshot: Integrate with snapuserd.
snapuserd: Add an API call to wait for device deletion.
This simple tool will dump the COW header and included ops to stdout.
Bug: N/A
Test: mm inspect_cow && inspect_cow <file>
Change-Id: I369c4a21a84c95ffc10670bd9eeb2ceccb2a56d6
Bug: 168554689
Test: vts_libsnapshot_test
Test: full OTA with update_device.py
Test: incremental OTA with update_device.py
Change-Id: I3878abfd767d2e47cf8486bc2c06233da2f1ef08
This integrates libsnapshot with dm-user and snapuserd. Tests progress
significantly further now. Tests involving merging still fail as
snapuserd doesn't support this yet.
Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: I464b683b464fe29a646f0f2823b7f4434a878614
This is in preparation for moving to a traditional client/server model
where clients stay connected and the server multiplexes multiple
connections.
Client has been renamed to DmUserClient to differentiate it from local
socket clients.
poll() responsibilities have been moved into SnapuserdServer. In
addition, the server now tracks all open clients and polls them
together with the listen socket.
SnapuserDaemon is now only responsible for signal masking. These two
classes can probably be merged together - I didn't do that here because
the patch was already large.
Bug: 168554689
Test: manual test
Change-Id: Ibc06f6287d49e832a8e25dd936ec07747a1b0555
This switches up the format to alternate ops with data, followed by a
footer containing additional meta information. This allows the file to
be resumed at arbitrary points if writing gets interrupted by power
loss.
Also adds a label op, which allows labeling future ops as connected.
If the footer is missing, Append will treat the last label as possibly
corrupt, and ignore it.
Change-Id: I126e15837d710776f9396e7afc9b0cd595e26b59
Bug: 168829493
Test: cow_api_test
This does not implement OpenReader() yet, however, it implements enough
for some vts_libsnapshot_tests to start passing with Virtual A/B
Compression enabled.
Note that OpenSnapshotWriter() has been disabled when linking with init
or recovery. This is to avoid pulling in all the compression libraries
in those places. OpenSnapshotWriter is only designed to be called by
update_engine and will not work outside of normal operation.
Bug: 168554689
Test: vts_libsnapshot_test
Test: full OTA with update_device.py
Test: incremental OTA with update_device.py
Change-Id: I9737d28bdd5c5f4914bc30a2bb72f357d5f44d2b
Bug: 168554689
Test: vts_libsnapshot_test
Test: full OTA with update_device.py
Test: incremental OTA with update_device.py
Change-Id: Ic5eb99be8ab1f89070a0db231d4660e123ae3967
This is a re-landing of the original CL, with a few changes:
- The correct device is now returned in MapUpdateSnapshot.
- The old API is used for tests, and the new API is only tested when
used on a VABC device.
- A sync() call has been added to ensure that writes to the base and
target snapshot devices have been fully flushed. This makes
IsPartitionUnchanged detect the MapUpdateSnapshot bug.
Implement OpenSnapshotWriter for non-compressed Virtual A/B. This is
done by adding an OnlineKernelSnapshotWriter class, which forwards all
writes to a dm-snapshot block device.
This also introduces a new ISnapshotWriter class which extends
ICowWriter, and adds features specific to libsnapshot (versus ICowWriter
which is intended only for the new COW format). The OpenSnapshotReader
call has been moved here since the writer retains all the information
needed to create the reader.
To test the new call, vts_libsnapshot_test has been modified to use
OpenSnapshotWriter.
As part of this change, all consumers of libsnapshot must now link to
libsnapshot_cow.
Bug: 168554689
Test: vts_libsnapshot_test
Test: full OTA with update_device.py
Test: incremental OTA with update_device.py
Change-Id: I90364a58902a4406a37cb14a816642c57a72bec2
Revert "Link to libsnapshot_cow everywhere libsnapshot is linked."
Revert submission 1433573-vab-libsnapshot-linkage
Reason for revert: b/169981170, update crash for droidfooders.
Reverted Changes:
Ie75bba98c:Link to libsnapshot_cow where libsnapshot is linke...
Ieedfadc55:libsnapshot: Partially implement OpenSnapshotWrite...
I28a5d4a88:Link to libsnapshot_cow everywhere libsnapshot is ...
Exempt-From-Owner-Approval: Revert to unblock dogfood
Change-Id: I0677df77672aca9fd54d94e009ac0be7c88a1a9d
1: Create a static library which exposes APIs
to manage snapuserd daemon.
2: Snapuserd daemon creates communication socket.
Bug: 168258493
Test: cow_snapuserd_test tests all the library API
along with the IO path from dm-snapshot.
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I8aaedf89d75e793c145fdad248a4d88e0ce8348c
Implement OpenSnapshotWriter for non-compressed Virtual A/B. This is
done by adding an OnlineKernelSnapshotWriter class, which forwards all
writes to a dm-snapshot block device.
This also introduces a new ISnapshotWriter class which extends
ICowWriter, and adds features specific to libsnapshot (versus ICowWriter
which is intended only for the new COW format). The OpenSnapshotReader
call has been moved here since the writer retains all the information
needed to create the reader.
To test the new call, vts_libsnapshot_test has been modified to use
OpenSnapshotWriter.
As part of this change, all consumers of libsnapshot must now link to
libsnapshot_cow.
Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: Ieedfadc557833c1e0540922aabc6e95c80266a64
