Commit graph

75873 commits

Author SHA1 Message Date
Kelvin Zhang
1fcdce21d6 Add zhangkelvin@ as owner of fastboot
Working on replacing aio w/ liburing in fastboot recently

Change-Id: I8076e2e53dc8d620faf9bff7f37ba2d2538460d6
2022-07-12 11:05:33 -07:00
Akilesh Kailash
58f66339ef Merge "libsnapshot: Fix vts_libsnapshot_test.SnapshotUpdateTest#FullUpdateFlow" 2022-07-12 17:53:09 +00:00
Treehugger Robot
c29f0745d5 Merge "Convert fastboot/libsnapshot to new BootControl client" 2022-07-12 17:47:20 +00:00
Sandeep Dhavale
f607e37fd0 Merge "Print OTA merge progress state in logs" 2022-07-12 17:12:23 +00:00
Pete Bentley
00bf9dcb01 Merge "Move boringssl self tests from early-init to init." 2022-07-12 09:50:40 +00:00
Treehugger Robot
eebd7f2b7d Merge "Do not enforce EROFS for android-T and below." 2022-07-12 06:39:59 +00:00
Akilesh Kailash
bfbd9b0e6e libsnapshot: Fix vts_libsnapshot_test.SnapshotUpdateTest#FullUpdateFlow
Check if compression feature is enabled.

Bug: 238143011
Test: OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I77663799a5371f1de8b4e29d7c777a24249d1b7b
2022-07-12 05:23:48 +00:00
Sandeep Dhavale
5bdb9dbb79 Print OTA merge progress state in logs
Currently OTA merge process prints enum values in logs in numerical
form. This patch changes the log messages from numerical enums to
more friendly names.

Test: Manual OTA

===========Logs==========
CheckTargetMergeState for vendor_dlkm_b returned: Merging
ProcessUpdateState handling state: Merging
CheckTargetMergeState for product_b returned: Merging
CheckTargetMergeState for vendor_dlkm_b returned: MergeCompleted
ProcessUpdateState handling state: Merging
CheckTargetMergeState for product_b returned: MergeCompleted
ProcessUpdateState handling state: MergeCompleted

Bug: 234518211
Change-Id: If690d3ea86e6fe37e8b3e4bd52e87b70ae92495c
2022-07-12 03:40:11 +00:00
Treehugger Robot
932e069f0f Merge "Don't kill subcontext if it's null" 2022-07-11 06:47:19 +00:00
Inseob Kim
5caaa5575e Don't kill subcontext if it's null
We don't initialize subcontext in microdroid. subcontext->pid() will
crash in such case.

Test: atest MicrodroidTests
Change-Id: Ifca65acededbf7f113d8bf16afbb8e2c1aa91b41
2022-07-11 14:34:15 +09:00
Kelvin Zhang
6befe78c3c Convert fastboot/libsnapshot to new BootControl client
Thew new client will use AIDL if available, and fallback to HIDL
otherwise.

Test: th
Bug: 227536004
Change-Id: I9af21037a76a4a6db00144f5b2774ea23f3a5cc2
2022-07-08 10:57:53 -07:00
P.Adarsh Reddy
a571d4a9bc Do not enforce EROFS for android-T and below.
EROFS is not mandatory for android T and below,
so skip the test for those.

Bug: 237765186
Test: vts_fs_test fs#ErofsSupported
Change-Id: Iceea46f8f2d443636de504962b718a2461605591
2022-07-08 12:14:01 +00:00
Pete Bentley
c017e2ce0a Move boringssl self tests from early-init to init.
In previous releases, these self tests had a secondary purpose
of writing a flag file to save future processes from running
some slow self checks.  This is no longer true in T.

However running the tests from early-init has caused issues
on some devices as the kernel's entropy pool is not yet
initialised, causing the process to block for a second or more.

Bug: 231946889
Test: m && flashall
Change-Id: I2116f2029ca6a21e4359407dfff4dc79edd39084
2022-07-07 15:34:46 +01:00
Treehugger Robot
ff3fdf36ef Merge "Check userspace snapshots only for API level >= T" 2022-07-06 08:59:12 +00:00
Benergy Meenan Ravuri
7680c29bc9 Check userspace snapshots only for API level >= T
Skip checking for userspace snapshots enabled property
for API level < T as this feature is not applicable for
GRF targets.

Bug: 236450435
Test: vts_ota_config_test
Change-Id: Ib5083f6237cdf4962aae06f166811d67cf6c385e
2022-07-05 16:43:31 +05:30
Yi-yo Chiang
827389c9a4 Merge "fs_mgr_fstab: Refactor & cleanup DSU mounting logic" 2022-07-04 05:32:51 +00:00
Jason Chiu
23ff619088 Merge "bootstat: add more bootreasons" 2022-07-03 14:07:17 +00:00
Jason Chiu
17251023bd bootstat: add more bootreasons
add more bootreasons for new projects

Bug: 219902794
Test: trigger apc watchdog then "adb root; adb shell bootstat -p"
Signed-off-by: Jason Chiu <jasoncschiu@google.com>
Change-Id: I7e19c4d48fb3d5b1c49dc8688936cf1d6eea6e9c
2022-07-03 14:07:09 +00:00
Akilesh Kailash
5c00853601 Merge changes I6791dd72,I887d5073
* changes:
  Reduce priority of merge threads
  Tune snapshot-merge performance
2022-07-03 06:54:12 +00:00
Akilesh Kailash
1e46611c78 Reduce priority of merge threads
Currently, when daemon is spin up, it runs at the highest
priority with nice value set to -20. This can potetially
lead to a problem in a busy system especially after OTA
reboot when all the merge threads are running in parallel.

Now that we reduced the number of merge threads in-flight
to 2, we reduce the priority as well by setting the nice
value to -5. The other threads which serve I/O's
from dm-user (from root filesystem) still runs at higher
priority. We need this because post OTA reboot, these
threads serve I/O's until merge is completed.

Merge threads on the other hand can run at a relatively
lower priority. We need to make sure that there
is always forward progress even in a busy system
and hence set the priority to -5 as compared
to default value of 0.

No boot time regressions observed.

Output of NICE value of merge and worker threads post OTA reboot:

1 S     0   427   451     1 0  39 -20  64 2314640 dev_r+ ?      00:00:00 8
1 S     0   427   486     1 4  39 -20  64 2314640 dev_r+ ?      00:00:02 8
1 S     0   427   487     1 4  39 -20  64 2314640 dev_r+ ?      00:00:02 8
1 S     0   427   488     1 3  39 -20  64 2314640 dev_r+ ?      00:00:02 8
5 R     0   427   634     1 1  24  -5  64 2314640 0    ?        00:00:00 8
5 R     0   427   935     1 5  24  -5  64 2314640 0    ?        00:00:02 8

Bug: 237490659
Test: Full and incremental OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I6791dd72ccd8cd5bba6eff663bb3f9598bce7ed2
2022-07-01 19:11:24 +00:00
Akilesh Kailash
9df7c07e5a Tune snapshot-merge performance
Currently, there is one thread per partition
for snapshot merge. When all these threads are
run in parallel, this may stress the system
as the merge threads are both CPU and I/O bound.

Allow only two merge threads to be in-flight
at any point in time. This will ensure that there
is forward progress done with respect to snapshot-merge
and only two cores are used as against using
5-6 cores.

Additionally, system and prodcut partitions are merged
first. This is primarily because /root is mounted
of system partition and faster the merge completes
on /system partition, we can switch the dm tables
immediately. There is no change in the merge phase
from libsnapshot perspective. This prioritization
is based on each merge phase. If the system partition
merge is in second phase, then it takes priority
in that phase.

As a side benefit, this should also
reduce the memory usage when merge is in-flight
given that we now limit the threads.

There is slight delay in overall merge time as
we now throttle the merge.

No boot time regressions observed.

Full OTA:

Merge time (Without this patch): 42 seconds
Merge time (With this patch): 46 seconds

Incremental OTA:

Merge time (Without this patch): 52 seconds
Merge time (With this patch): 57 seconds

system partition merge completes in the first ~12-16 seconds.

App-launch (COLD) on Pixel:

Baseline (After snapshot-merge is completed when there is no daemon):
==========================

Chrome: 250
youtube: 631
camera: 230

==========================

Without this patch when snapshot-merge is in-progress (in ms):

Full - OTA

Chrome: 1729
youtube: 3126
camera: 1525

==========================

With this patch when snapshot-merge is in-progress (in ms):


Full - OTA

Chrome:  1061
youtube: 820
camera: 1378

Incremental - OTA (350M)

Chrome: 495
youtube: 1442
camera: 641
=====================

Bug: 237490659
Test: Full and incremental OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I887d5073dba88e9a8a85ac10c771e4ccee7c84ff
2022-07-01 19:05:09 +00:00
Peter Collingbourne
73583331a0 Merge "Fix scudo MTE tests." 2022-07-01 17:09:55 +00:00
Peter Collingbourne
7827991d7f Fix scudo MTE tests.
r.android.com/2108505 was intended to fix a crash in Scudo in
the case where the stack depot, region info or ring buffer were
unreadable. However, it also ended up introducing a number of bugs into
the code. It failed to call __scudo_get_error_info if the page at the
fault address was unreadable. This can happen in legitimate crash cases
if a primary allocation was close to the boundary of a mapped region,
or if the allocation was a secondary allocation with guard pages. It
also used long as the type for tags, whereas Scudo expects it to be
char. In combination this ended up causing most of the MTE tests to
fail. Therefore, mostly revert that change.

Fix the original crash by null checking the pointers returned by
AllocAndReadFully before proceeding with the rest of the function.

Bug: 233720136
Change-Id: I04d70d2abffaa35fe315d15d9224f9b412a9825d
2022-06-30 18:54:19 -07:00
Yi-Yo Chiang
f8671e0fc1 fs_mgr_fstab: Refactor & cleanup DSU mounting logic
* Instead of copy-and-modify `/data` mount entry, TransformFstabForDsu()
  can just modify all `/data` mount entries in-place.
* This also stops TransformFstabForDsu() from shuffling the mount entry
  of `/data` in fstab, simplifying the testcase.
* Implement GetEntriesForMountPoint() with GetEntriesByPred().
* Remove unused method BuildDsuUserdataFstabEntry() and
  EraseFstabEntry().

Bug: 235111004
Test: atest CtsFsMgrTestCases
Test: Presubmit GSI boot test
Change-Id: I4afb443b7e527038fb42424543bb1538b01d5ec9
2022-06-30 06:46:02 +00:00
Yi-yo Chiang
fad8eabad7 Merge "fs_mgr_fstab: Remove legacy DSU logic for Q first_stage_init" 2022-06-30 06:07:35 +00:00
Yi-Yo Chiang
e0cd16ba41 fs_mgr_fstab: Remove legacy DSU logic for Q first_stage_init
Deprecate DSU booting support on Q first_stage_init.
Now kDsuActiveFile and kGsiLpNamesFile must be accessible for
ReadFstabFromFile() to return successfully.

Bug: 235111004
Test: Presubmit GSI boot test
Change-Id: I5760eb9aa05610085dd7b7b78940f22fba425229
2022-06-30 06:07:29 +00:00
Treehugger Robot
3854a69d39 Merge "Add lz4 support for VABC" 2022-06-30 00:15:42 +00:00
Kelvin Zhang
5cb1b0295e Add lz4 support for VABC
Add lz4 support in CowWriter/CowReader. Lz4 should offer faster
read/write performance but slightly larger COW sizes.

Download time: Reduced from 1264s to 825s (~35% faster)
Filesystem verification time: from 69s to 59s (~15% faster)
COW Size: 2.59GB to 3.21GB, (~25% regression)
Merge time: 37046ms to 27690ms (~25% faster)
Boot time: LZ4 is 2.7% faster

Overall, LZ4 offers faster read/write performance at the cost of 25%
space regression.

Test: th
Bug: 228478555
Change-Id: Ie521852fb3c9ff8a12e55b764c1eb1838f7b2181
2022-06-29 13:10:55 -07:00
David Anderson
4a2beb8d6a Merge "EXT4_FEATURE_COMPAT_STABLE_INODES from ext4.h" 2022-06-28 18:14:28 +00:00
Treehugger Robot
13faa1b394 Merge "first_stage_init: Optimize redundant vector copy" 2022-06-28 05:48:17 +00:00
Steven Moreland
1cdb22fb05 Merge "RefBase: disallow make_shared, make_unique" 2022-06-27 18:14:52 +00:00
Yi-Yo Chiang
01a7dcb240 first_stage_init: Optimize redundant vector copy
```
std::vector<std::string> v2;
for (auto&& e : get_temporary_v1()) { v2.push_back(e); }
```
^^^ This is constructing v2 from a temporary object, which is an
unnecessary manual clone. We should just let copy-elision do its thing:

```
// Give compiler the chance to optimize this with copy-elision...
auto v2 = get_temporary_v1();
```

Also `lp_names` appends an extra ',' at its end. Just use
android::base::Join() here.

Bug: 235111004
Test: Presubmit GSI boot test
Change-Id: Ibe8ce7a29b8521e789aa0a99f7f6d31f3f9c70e9
2022-06-27 16:19:37 +00:00
Yi-yo Chiang
e8e7b3bb26 Merge "TransformFstabForDsu: Support erofs GSI and refactor" 2022-06-27 03:59:23 +00:00
jiajia tang
dc07763585 EXT4_FEATURE_COMPAT_STABLE_INODES from ext4.h
Include EXT4_FEATURE_COMPAT_STABLE_INODES feature
flag from system/extra.

Test: m
Test: mmma system/core

Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
Change-Id: I85f4a1c66cdfb37246808d58ca6315861daafe83
2022-06-25 10:07:54 +00:00
David Anderson
afffa03f24 Merge "fastboot: Add a command to get current kernel logs." 2022-06-24 21:10:55 +00:00
Yi-Yo Chiang
c46c89228c TransformFstabForDsu: Support erofs GSI and refactor
Synthesis mount entry for erofs and ext4.
Fix the synthesis logic to always create a new entry from scratch. The
old logic of "copy an existing mount entry and edit its fs_type"
doesn't work because different fs_type would have incomptable
fs_options. For example, changing this:
  system  /system  ext4  ro,barrier=1  wait,logical,first_stage_mount
to this:
  system  /system  erofs  ro,barrier=1  wait,logical,first_stage_mount
doesn't work, because the erofs driver won't recognize the "barrier=1"
mount flag, since "barrier=1" is specific to ext4.

Bug: 235111004
Test: Boot erofs GSI.
Test: atest CtsFsMgrTestCases
Change-Id: I57132a55a089c7aae3e17c717ecd9dc1047fede8
2022-06-24 16:47:36 +08:00
Treehugger Robot
3be42ae576 Merge "Do not create /data/misc/zoneinfo." 2022-06-23 17:41:39 +00:00
Almaz Mingaleev
2f38c39169 Do not create /data/misc/zoneinfo.
APK time zone update mechanism used to store tzdata file there.
The feature is removed, no need to create that folder.

Bug: 148144561
Test: atest CtsBionicTestCases
Test: atest BionicTzdbConsistencyTest

Change-Id: I249f1d1b6c1a3f1a283d1ca43fcc93b10cbd910a
2022-06-23 17:21:39 +01:00
Treehugger Robot
d9a4563c53 Merge "Rewrite function declarators without prototypes" 2022-06-23 08:03:58 +00:00
Almaz Mingaleev
7f79ee42ee Merge "Remove TZUvA feature." 2022-06-23 07:47:26 +00:00
Steven Moreland
9d8af6af65 RefBase: disallow make_shared, make_unique
It's risky to mix multiple-ownership types. Taken from SharedRefBase.

Bug: 232557259
Test: N/A
Change-Id: Ic0dbd6d11e44fa9db87c4f9b1776d4989cbf9f56
2022-06-22 21:54:43 +00:00
Eric Biggers
e5a9a70437 Merge "Rename fscrypt_is_native() to IsFbeEnabled()" 2022-06-22 18:24:38 +00:00
Florian Mayer
ff5834a3e6 Merge "Promote HWASan tests to presubmit." 2022-06-22 17:55:41 +00:00
Yi Kong
d82c4e71c9 Rewrite function declarators without prototypes
This syntax is removed in C2x proposal N2841.

Test: presubmit
Change-Id: I3b067023327ed22dffda698c5f1accde60d95839
2022-06-22 16:59:22 +08:00
David Anderson
ebc8fe19f8 fastboot: Add a command to get current kernel logs.
This adds "fastboot getvar dmesg", usable only in userspace fastboot,
and only on unlocked devices. This is to debug flashing failures when
serial log access is not available.

Bug: 230269532
Test: fastboot getvar dmesg
Change-Id: Ic57881e362efe3f0687f41ab986d30b3a59dc4e4
2022-06-21 19:32:19 -07:00
Treehugger Robot
8874ad6378 Merge "Promote HWASan tests to presubmit." 2022-06-21 17:32:43 +00:00
Treehugger Robot
03d309beb2 Merge "Promote HWASan tests to presubmit." 2022-06-21 16:11:21 +00:00
Florian Mayer
556b4e0f80 Promote HWASan tests to presubmit.
They have been passing and do not show significant slowdown.

Change-Id: I8f6ac751515cffd4904e10d1ec473c3a1fff60ec
2022-06-17 22:41:33 +00:00
Florian Mayer
4effd2f901 Promote HWASan tests to presubmit.
They have been passing and do not show significant slowdown.

Change-Id: I3eba6ecb746160d75e9a2c9561bceb71a09117d3
2022-06-17 22:29:09 +00:00
Florian Mayer
db5b2c9fc0 Promote HWASan tests to presubmit.
They have been passing and do not show significant slowdown.

Change-Id: I64328118f0606c1b62a793fb8913eed5e5ba9578
2022-06-17 22:18:49 +00:00