Commit graph

1521 commits

Author SHA1 Message Date
Tom Cherry
aef269f11b Merge changes from topic "logwrapper-api-update"
* changes:
  Update init/fs_mgr for new logwrapper function
  logwrap: convert to C++, rename function logwrap_fork_execvp().
2019-09-27 15:31:04 +00:00
Yifan Hong
3edff134ef Merge "libsnapshot: refactor CreateUpdateSnapshots" 2019-09-27 01:11:22 +00:00
Tom Cherry
3a803eb2d4 Update init/fs_mgr for new logwrapper function
Test: cuttlefish boots, fsck logs seen
Change-Id: Icbc01161b44e097be813f4af648fa671739078b9
2019-09-26 10:29:57 -07:00
Colin Cross
2a62d42704 Merge "Use libcrypto_static instead of libcrypto" 2019-09-26 14:32:33 +00:00
Yifan Hong
98aa26fe69 libsnapshot: refactor CreateUpdateSnapshots
This function is too big. Refactor it a bit.

- Reorder checks / variable initializations a bit.
  They are independent of each other, so they can be
  safely reordered.
- Put snapshot creation and initialization to their own
  functions so that CreateUpdateSnapshots looks shorter.

Test: libsnapshot_test
Change-Id: I135b415d8e046dd91d31326fe7962ae44be4ccf8
2019-09-25 15:20:50 -07:00
Yifan Hong
ab09a1764b Merge changes Iab867ded,Id37e34b4
* changes:
  libsnapshot: CreateUpdateSnapshots reuse COW space
  libsnapshot: fix tests
2019-09-25 22:19:50 +00:00
Yifan Hong
bc71949dd7 Merge "libsnapshot: Fix mocked property fetcher" 2019-09-25 22:19:35 +00:00
Treehugger Robot
9e8f01101f Merge "liblp: convert more functions to use string_view (#5)" 2019-09-25 20:49:36 +00:00
Yifan Hong
30bdee9910 libsnapshot: CreateUpdateSnapshots reuse COW space
At the second update for Virtual A/B device,
PartitionCowCreator thinks the COW partitions in
the existing super metadata as occupied. In fact, these
partitions aren't used after the merge.

Now, unmap these partitions from the device mapper. If
something bad happens and the previous update has not been
merged yet, the unmap will fail. Then, delete these old
COW partitions from the device.

Test: Virtual A/B update twice
Bug: 135752105
Change-Id: Iab867ded19755089e6e0800e553a10fbcddbb931
2019-09-25 12:13:17 -07:00
Yifan Hong
3396e9327a libsnapshot: fix tests
- Add TestDeviceInfo(string, string) to avoid holding
the 'rebooted' TestDeviceInfo variable after ownership is
transferred

- OTA client calls BeginUpdate after unmapping partitions.

Test: run libsnapshot_test
Change-Id: Id37e34b421d728eb320c0e0906abef3b61897e7c
2019-09-25 12:13:17 -07:00
Yifan Hong
30adb65103 libsnapshot: Fix mocked property fetcher
Add a new SnapshotTestPropertyFetcher that sets up
good defaults for all test cases in libsnapshot_test.

Test: run libsnapshot_test
Change-Id: I6e0b31872d604bbd5f15b64ab7a8ea56d7c67405
2019-09-25 12:05:53 -07:00
Treehugger Robot
b4a65cf1ec Merge "[vts-core] Add vts_core_liblp_test to vts-core" 2019-09-25 05:54:31 +00:00
Treehugger Robot
5781a185e1 Merge "[vts-core] Add libdm_test in vts-core" 2019-09-25 03:00:50 +00:00
Yifan Hong
a1c1eb604d liblp: convert more functions to use string_view (#5)
Test: builds

Change-Id: I59ef370489b36410e3eaf90dc2623efaeefd613b
2019-09-24 19:34:54 -07:00
Treehugger Robot
8c055fbe3e Merge "snapshotctl: merge when boot" 2019-09-24 20:13:52 +00:00
Yifan Hong
71667636fb Merge "libsnapshot: add snapshotctl" 2019-09-24 18:06:53 +00:00
nelsonli
5642e23949 [vts-core] Add libdm_test in vts-core
Add libdm_test to vts-core suite and skip the test if the
first-api-level < 29.

Bug: 132702215
Test: atest vts_libdm_test
Change-Id: Ic805a4c08e5bf9f6c69019d4ea01825ae888e1fe
2019-09-24 11:34:59 +08:00
Yifan Hong
8a679e673f snapshotctl: merge when boot
Test: reboot after OTA on virtual A/B device, see merge
Bug: 135752105

Change-Id: I92eb8d0961635008366c336ead40e3038173257f
2019-09-23 18:50:12 -07:00
Yifan Hong
2feb47a29b libsnapshot: add snapshotctl
... which handles merge when boot completed. It also
dumps debug information when requested.

Bug: 135752105
Test: adb shell su 0 snapshotctl dump
Test: call snapshotctl merge at different stage of OTA:
  - before OTA (exit normally)
  - during OTA is applied (exit with error)
  - after OTA is applied but before reboot (exit with error)
  - after reboot (finish merge and exit normally)
  - Manually stop its execution during merge and run again
      (finish merge and exit normally)

Change-Id: Idcc6aea8d7bbeb9a1a288c966b8f5e14b3f6a3e7
2019-09-23 18:50:12 -07:00
Howard Chen
e6d71689bb Merge "Mount multiple DSU partitions when present." 2019-09-21 02:48:37 +00:00
Yifan Hong
e71efc3dc4 Merge changes from topic "libsnapshot_api_update_engine"
* changes:
  libsnapshot: Cancel/merge existing update before begin
  libsnapshot: operator<< for UpdateState
2019-09-20 20:56:50 +00:00
nelsonli
2e9edcb042 [vts-core] Add vts_core_liblp_test to vts-core
Add vts_core_liblp_test to vts-core suite and skip the test if the
first-api-level < 29.

Bug: 132702215
Test: atest vts_core_liblp_test
Change-Id: I09ea8ff923b5300bae513146f6d07dc029d36661
2019-09-20 16:06:18 +08:00
Howard Chen
1b094939e5 Mount multiple DSU partitions when present.
There might be partitions like product or system_ext in addition
to the system partition. Those partitions can contain dependencies
required by the system so we need to make the init to recognize
these partitions and mount them accordingly.

Bug: 140092208
Test: gsi_tool install & reboot
Change-Id: Ie5ef063292c3656e79cce9f887f23a8faa1ba2be
2019-09-20 14:10:52 +08:00
Treehugger Robot
da6525c2db Merge "Construct the super_vbmeta image" 2019-09-19 10:21:53 +00:00
Kaiwen Szu
6dd098cb1e Construct the super_vbmeta image
This commit constructs the super_vbmeta image to eliminate the
need of adding /vbmeta_system and/or /vbmeta_vendor when AVB
chain partition is used with Android Dynamic Partition.

See BOARD_AVB_VBMETA_SYSTEM under the link:
https://android.googlesource.com/platform/external/avb/#build-system-integration

The structure of super_vbmeta :

|     VBMeta Table    | (fixed-length 2KiB)
| Backup VBMeta Table | (fixed-length 2KiB)
|     VBMeta Images   | (fixed-length 64KiB each)

The structure of VBMeta Table :

| Super VBMeta Header | (fixed-length 128B)
|  VBMeta Descriptors | (variable-length)

The VBMeta Table records the slot number of each
vbmeta image within the /super_vbmeta partition.

Bug: 137054296
Test: m libvbmeta_test
Test: ./out/host/linux-x86/nativetest/libvbmeta_test/libvbmeta_test
Change-Id: I01aeadd850750ae87d9125484c1b1f570bb84756
2019-09-19 12:05:09 +08:00
Colin Cross
f117f34aed Use libcrypto_static instead of libcrypto
Replace libcrypto with libcrypto_static, which can be protected through
visibility to ensure only modules that don't affect FIPS certification
can use it.

Bug: 141248879
Test: m checkbuild
Change-Id: I5f0b9acfb57f68570f6f58f2395f2bb1bc015365
2019-09-18 11:04:35 -07:00
Yifan Hong
6aa77dd271 libsnapshot: Cancel/merge existing update before begin
Attempt to cancel or merge an existing update before starting
a new one.

Test: libsnapshot_test

Change-Id: Ic534db846f1fe97bd7ca1f176a9494fbc6a0f8d5
2019-09-17 16:58:34 -07:00
Yifan Hong
0b32b199ac libsnapshot: operator<< for UpdateState
Test: pass
Change-Id: Ifc88666d0b7c76f3ea9b7ec662398084d631c4e6
2019-09-17 16:58:34 -07:00
Yifan Hong
184bb8a77f libsnapshot: CreateUpdateSnapshot accepts DeltaArchiveManifest
DeltaArchiveManifest includes full update package metadata.
Now, CreateUpdateSnapshot reads the source
metadata, updates it, and write to target metadata slot as well.

Test: libsnapshot_test
Bug: 140868231

Change-Id: Ia885b336145d02111ecff1aad421cb9b1efd18c2
2019-09-17 12:55:23 -07:00
Yifan Hong
d22459dff3 libsnapshot: Add snapshot metadata updater.
This class knows how to update super metadata for virtual A/B devices.

Test: libsnapshot_test
Bug: 138816109

Change-Id: I9e375c76814e0dcbb47fc2ea9e4903ba69ccf7f8
2019-09-17 12:55:23 -07:00
Yifan Hong
116d2ff503 Merge "liblp: convert more functions to use string_view (#4)" 2019-09-16 21:56:09 +00:00
Yifan Hong
c2ff970b17 liblp: convert more functions to use string_view (#4)
Test: builds
Change-Id: Iad041762beb9eeab8b2174ab48a50677754248b5
2019-09-16 14:50:40 -07:00
Treehugger Robot
17438477cb Merge "libsnapshot: Fix typo in nullptr check" 2019-09-16 04:52:31 +00:00
Greg Kaiser
99ab97836b libsnapshot: Fix typo in nullptr check
We already know 'existing_extent' is non-null (or else we would
have crashed on the previous line).  We fix this to check
'existing_linear_extent' against nullptr.

Test: TreeHugger
Change-Id: I03338bca87d1c2bd0cbea0e1ec31244ba9570a40
2019-09-13 06:58:03 -07:00
Yifan Hong
f6fccc466a Move digital_stroage.h to its own library
...so that it can be used by other tests.

Also, clean up the code before exposing it.

Test: libsnapshot_test
Test: liblp_test

Change-Id: I627326f696ea55b7113ff26b313f7dd04e341dc1
2019-09-12 15:50:51 -07:00
Yifan Hong
26d7d95391 libsnapshot: tests for public APIs.
Test: libsnapshot_test

Change-Id: I411ae32e77914845ed4037d7e67620598f8218cf
2019-09-11 18:32:57 -07:00
Yifan Hong
0e13bbade3 fastbootd: skip COW group
Skip importing COW group because they are dynamically
created by OTA clients and will never show up in built
images.
Test: flash

Change-Id: I44e7693cfb4c15e64455b56212a02f9abde88d61
2019-09-11 18:32:57 -07:00
Yifan Hong
65d94596f2 libsnapshot: no overlayfs during virtual a/b ota.
With overlayfs enabled, the scratch partition takes up
super partition, causing retrofit devices to create COW
images.
Test: OTA on retrofit device, no COW images created.

Change-Id: I35a1a2d02c8acf9efc33965c7e015f960bf45296
2019-09-11 18:32:57 -07:00
Yifan Hong
20a23bb41c libsnapshot: Also use empty space in super for COW
The super partition usually has some empty space even after
the target partitions are created, especially for retrofit
Virtual A/B devices. Use that empty space for COW before taking
up userdata space.

- PartitionCowCreator computes free regions in super partition metadata
  and use that space until it is used up. It returns a pair of numbers
  (cow_partition_size, cow_file_size) and let SnapshotManager to create
  the partition / images with proper sizes.
  - A region is considered free iff it is used by NEITHER target NOR
    source partitions
  - The list is in PartitionCowCreator's return value so that
    SnapshotManager can use it as a guide when creating partitions.
  - These partitions are created under the group named "cow".
    - Avoid mapping COW partitions directly during init first stage
      mount. Init only maps them when they are needed by the top-level
      device.
- CreateCowImage no longer zero-fills the first 4 bytes of the image.
  (See below)
- CreateUpdatePartitions: after creating the snapshot, also maps the COW
  devices (via MapCowDevices()) and zero-fills the first 4 bytes of the
  top-level COW device.
- Add a new SnapshotManager::MapCowDevices() function, which maps both
  the COW partition (in super) and the COW image (through
  IImageManager) (if necessary). Then, create a dm-linear target that
  concatenates them (if necessary).
- Add a new SnapshotManager::UnmapCowDevices() functions that does the
  reverse MapCowDevices().
- DeleteSnapshot also unmaps the top-level COW device and COW partition
  before unmapping the COW images (before deleting them).

Test: libsnapshot_test

Change-Id: I0098b7e842ab48b0b4dd2a59142098b098d23d34
2019-09-11 18:32:57 -07:00
Yifan Hong
c69029f2a8 libsnapshot: APIs for all partitions
Add CreateCowForUpdate / MapSnapshotDevicesForUpdate
that update_engine and init can call them directly.

Bug: 134536978
Test: libsnapshot_test

Change-Id: If53c48855931db27454fd2893745915c77fd37f8
2019-09-11 18:32:57 -07:00
Yifan Hong
c02509b500 fs_mgr: CreateDmTable takes CreateLogicalPartitionParams
It has too many arguments.

Also fixed CreateLogicalPartitionParams::InitDefaults because
it doesn't use the provided partition opener to read metadata
(which breaks tests).

Test: libsnapshot_test
Test: liblp_test

Change-Id: I74cf8d468274f741c6f3743438fe8855b2aef15b
2019-09-11 18:32:57 -07:00
Yifan Hong
c42f5d64e4 Merge "libsnapshot: collapse fix sanity checks" 2019-09-10 23:47:15 +00:00
Yifan Hong
24815a49d4 Merge "libsnapshot: MapSnapshot: use device string for inner device" 2019-09-10 23:46:22 +00:00
David Anderson
878930be92 Merge "liblp: Add a helper to differentiate empty from non-empty images." 2019-09-10 21:36:39 +00:00
Yifan Hong
af32f9377e libsnapshot: collapse fix sanity checks
Fix sanity checks of the snapshot device sizes.
Test: libsnapshot_test

Change-Id: If2945eaa84be59e14ea1fbc9028022db3a145698
2019-09-09 19:45:14 -07:00
Yifan Hong
f4cd54e512 libsnapshot: MapSnapshot: use device string for inner device
Previously, the device path for the -inner device is used, which
may not be present. Use the major:minor instead.

Test: libsnapshot (not flaky)
Change-Id: I8b8cbe8af27a49a50876385bb0c0353ecf7ed1de
2019-09-09 19:45:14 -07:00
Yifan Hong
26a2e6eb7b libsnapshot: cancel ok if unverified and old slot
Allow canceling an update if the update has been applied
but not rebooted into it.

Test: libsnapshot_test
Change-Id: I694d74e200908ec622855074ab811e3029328f43
2019-09-09 14:41:57 -07:00
David Anderson
922cfefbb9 liblp: Add a helper to differentiate empty from non-empty images.
Also, finally explain the difference between these somewhere.

Bug: 140204341
Test: lpdump
Change-Id: I128780b01a28e893afac65f3aa4f4a8d36032d63
2019-09-09 13:16:02 -07:00
Yifan Hong
233c32a454 Merge changes from topic "libsnapshot_prep"
* changes:
  libsnapshot: MapPartitionWithSnapshot cleanup itself if failed
  libsnapshot: MapPartitionWithSnapshot: fix timeout
  libsnapshot: require ex lock for {Create}{CowImage,Snapshot} / DeleteSnapshot
  libsnapshot: Refactor: add {Create,Map,Unmap}CowImage
2019-09-06 19:28:09 +00:00
Treehugger Robot
0b18c6735f Merge "fs_mgr: overlayfs: README merge in public document" 2019-09-06 03:28:42 +00:00