Commit graph

75861 commits

Author SHA1 Message Date
Pete Bentley
c017e2ce0a Move boringssl self tests from early-init to init.
In previous releases, these self tests had a secondary purpose
of writing a flag file to save future processes from running
some slow self checks.  This is no longer true in T.

However running the tests from early-init has caused issues
on some devices as the kernel's entropy pool is not yet
initialised, causing the process to block for a second or more.

Bug: 231946889
Test: m && flashall
Change-Id: I2116f2029ca6a21e4359407dfff4dc79edd39084
2022-07-07 15:34:46 +01:00
Treehugger Robot
ff3fdf36ef Merge "Check userspace snapshots only for API level >= T" 2022-07-06 08:59:12 +00:00
Benergy Meenan Ravuri
7680c29bc9 Check userspace snapshots only for API level >= T
Skip checking for userspace snapshots enabled property
for API level < T as this feature is not applicable for
GRF targets.

Bug: 236450435
Test: vts_ota_config_test
Change-Id: Ib5083f6237cdf4962aae06f166811d67cf6c385e
2022-07-05 16:43:31 +05:30
Yi-yo Chiang
827389c9a4 Merge "fs_mgr_fstab: Refactor & cleanup DSU mounting logic" 2022-07-04 05:32:51 +00:00
Jason Chiu
23ff619088 Merge "bootstat: add more bootreasons" 2022-07-03 14:07:17 +00:00
Jason Chiu
17251023bd bootstat: add more bootreasons
add more bootreasons for new projects

Bug: 219902794
Test: trigger apc watchdog then "adb root; adb shell bootstat -p"
Signed-off-by: Jason Chiu <jasoncschiu@google.com>
Change-Id: I7e19c4d48fb3d5b1c49dc8688936cf1d6eea6e9c
2022-07-03 14:07:09 +00:00
Akilesh Kailash
5c00853601 Merge changes I6791dd72,I887d5073
* changes:
  Reduce priority of merge threads
  Tune snapshot-merge performance
2022-07-03 06:54:12 +00:00
Akilesh Kailash
1e46611c78 Reduce priority of merge threads
Currently, when daemon is spin up, it runs at the highest
priority with nice value set to -20. This can potetially
lead to a problem in a busy system especially after OTA
reboot when all the merge threads are running in parallel.

Now that we reduced the number of merge threads in-flight
to 2, we reduce the priority as well by setting the nice
value to -5. The other threads which serve I/O's
from dm-user (from root filesystem) still runs at higher
priority. We need this because post OTA reboot, these
threads serve I/O's until merge is completed.

Merge threads on the other hand can run at a relatively
lower priority. We need to make sure that there
is always forward progress even in a busy system
and hence set the priority to -5 as compared
to default value of 0.

No boot time regressions observed.

Output of NICE value of merge and worker threads post OTA reboot:

1 S     0   427   451     1 0  39 -20  64 2314640 dev_r+ ?      00:00:00 8
1 S     0   427   486     1 4  39 -20  64 2314640 dev_r+ ?      00:00:02 8
1 S     0   427   487     1 4  39 -20  64 2314640 dev_r+ ?      00:00:02 8
1 S     0   427   488     1 3  39 -20  64 2314640 dev_r+ ?      00:00:02 8
5 R     0   427   634     1 1  24  -5  64 2314640 0    ?        00:00:00 8
5 R     0   427   935     1 5  24  -5  64 2314640 0    ?        00:00:02 8

Bug: 237490659
Test: Full and incremental OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I6791dd72ccd8cd5bba6eff663bb3f9598bce7ed2
2022-07-01 19:11:24 +00:00
Akilesh Kailash
9df7c07e5a Tune snapshot-merge performance
Currently, there is one thread per partition
for snapshot merge. When all these threads are
run in parallel, this may stress the system
as the merge threads are both CPU and I/O bound.

Allow only two merge threads to be in-flight
at any point in time. This will ensure that there
is forward progress done with respect to snapshot-merge
and only two cores are used as against using
5-6 cores.

Additionally, system and prodcut partitions are merged
first. This is primarily because /root is mounted
of system partition and faster the merge completes
on /system partition, we can switch the dm tables
immediately. There is no change in the merge phase
from libsnapshot perspective. This prioritization
is based on each merge phase. If the system partition
merge is in second phase, then it takes priority
in that phase.

As a side benefit, this should also
reduce the memory usage when merge is in-flight
given that we now limit the threads.

There is slight delay in overall merge time as
we now throttle the merge.

No boot time regressions observed.

Full OTA:

Merge time (Without this patch): 42 seconds
Merge time (With this patch): 46 seconds

Incremental OTA:

Merge time (Without this patch): 52 seconds
Merge time (With this patch): 57 seconds

system partition merge completes in the first ~12-16 seconds.

App-launch (COLD) on Pixel:

Baseline (After snapshot-merge is completed when there is no daemon):
==========================

Chrome: 250
youtube: 631
camera: 230

==========================

Without this patch when snapshot-merge is in-progress (in ms):

Full - OTA

Chrome: 1729
youtube: 3126
camera: 1525

==========================

With this patch when snapshot-merge is in-progress (in ms):


Full - OTA

Chrome:  1061
youtube: 820
camera: 1378

Incremental - OTA (350M)

Chrome: 495
youtube: 1442
camera: 641
=====================

Bug: 237490659
Test: Full and incremental OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I887d5073dba88e9a8a85ac10c771e4ccee7c84ff
2022-07-01 19:05:09 +00:00
Peter Collingbourne
73583331a0 Merge "Fix scudo MTE tests." 2022-07-01 17:09:55 +00:00
Peter Collingbourne
7827991d7f Fix scudo MTE tests.
r.android.com/2108505 was intended to fix a crash in Scudo in
the case where the stack depot, region info or ring buffer were
unreadable. However, it also ended up introducing a number of bugs into
the code. It failed to call __scudo_get_error_info if the page at the
fault address was unreadable. This can happen in legitimate crash cases
if a primary allocation was close to the boundary of a mapped region,
or if the allocation was a secondary allocation with guard pages. It
also used long as the type for tags, whereas Scudo expects it to be
char. In combination this ended up causing most of the MTE tests to
fail. Therefore, mostly revert that change.

Fix the original crash by null checking the pointers returned by
AllocAndReadFully before proceeding with the rest of the function.

Bug: 233720136
Change-Id: I04d70d2abffaa35fe315d15d9224f9b412a9825d
2022-06-30 18:54:19 -07:00
Yi-Yo Chiang
f8671e0fc1 fs_mgr_fstab: Refactor & cleanup DSU mounting logic
* Instead of copy-and-modify `/data` mount entry, TransformFstabForDsu()
  can just modify all `/data` mount entries in-place.
* This also stops TransformFstabForDsu() from shuffling the mount entry
  of `/data` in fstab, simplifying the testcase.
* Implement GetEntriesForMountPoint() with GetEntriesByPred().
* Remove unused method BuildDsuUserdataFstabEntry() and
  EraseFstabEntry().

Bug: 235111004
Test: atest CtsFsMgrTestCases
Test: Presubmit GSI boot test
Change-Id: I4afb443b7e527038fb42424543bb1538b01d5ec9
2022-06-30 06:46:02 +00:00
Yi-yo Chiang
fad8eabad7 Merge "fs_mgr_fstab: Remove legacy DSU logic for Q first_stage_init" 2022-06-30 06:07:35 +00:00
Yi-Yo Chiang
e0cd16ba41 fs_mgr_fstab: Remove legacy DSU logic for Q first_stage_init
Deprecate DSU booting support on Q first_stage_init.
Now kDsuActiveFile and kGsiLpNamesFile must be accessible for
ReadFstabFromFile() to return successfully.

Bug: 235111004
Test: Presubmit GSI boot test
Change-Id: I5760eb9aa05610085dd7b7b78940f22fba425229
2022-06-30 06:07:29 +00:00
Treehugger Robot
3854a69d39 Merge "Add lz4 support for VABC" 2022-06-30 00:15:42 +00:00
Kelvin Zhang
5cb1b0295e Add lz4 support for VABC
Add lz4 support in CowWriter/CowReader. Lz4 should offer faster
read/write performance but slightly larger COW sizes.

Download time: Reduced from 1264s to 825s (~35% faster)
Filesystem verification time: from 69s to 59s (~15% faster)
COW Size: 2.59GB to 3.21GB, (~25% regression)
Merge time: 37046ms to 27690ms (~25% faster)
Boot time: LZ4 is 2.7% faster

Overall, LZ4 offers faster read/write performance at the cost of 25%
space regression.

Test: th
Bug: 228478555
Change-Id: Ie521852fb3c9ff8a12e55b764c1eb1838f7b2181
2022-06-29 13:10:55 -07:00
David Anderson
4a2beb8d6a Merge "EXT4_FEATURE_COMPAT_STABLE_INODES from ext4.h" 2022-06-28 18:14:28 +00:00
Treehugger Robot
13faa1b394 Merge "first_stage_init: Optimize redundant vector copy" 2022-06-28 05:48:17 +00:00
Steven Moreland
1cdb22fb05 Merge "RefBase: disallow make_shared, make_unique" 2022-06-27 18:14:52 +00:00
Yi-Yo Chiang
01a7dcb240 first_stage_init: Optimize redundant vector copy
```
std::vector<std::string> v2;
for (auto&& e : get_temporary_v1()) { v2.push_back(e); }
```
^^^ This is constructing v2 from a temporary object, which is an
unnecessary manual clone. We should just let copy-elision do its thing:

```
// Give compiler the chance to optimize this with copy-elision...
auto v2 = get_temporary_v1();
```

Also `lp_names` appends an extra ',' at its end. Just use
android::base::Join() here.

Bug: 235111004
Test: Presubmit GSI boot test
Change-Id: Ibe8ce7a29b8521e789aa0a99f7f6d31f3f9c70e9
2022-06-27 16:19:37 +00:00
Yi-yo Chiang
e8e7b3bb26 Merge "TransformFstabForDsu: Support erofs GSI and refactor" 2022-06-27 03:59:23 +00:00
jiajia tang
dc07763585 EXT4_FEATURE_COMPAT_STABLE_INODES from ext4.h
Include EXT4_FEATURE_COMPAT_STABLE_INODES feature
flag from system/extra.

Test: m
Test: mmma system/core

Signed-off-by: jiajia tang <tangjiajia@xiaomi.com>
Change-Id: I85f4a1c66cdfb37246808d58ca6315861daafe83
2022-06-25 10:07:54 +00:00
David Anderson
afffa03f24 Merge "fastboot: Add a command to get current kernel logs." 2022-06-24 21:10:55 +00:00
Yi-Yo Chiang
c46c89228c TransformFstabForDsu: Support erofs GSI and refactor
Synthesis mount entry for erofs and ext4.
Fix the synthesis logic to always create a new entry from scratch. The
old logic of "copy an existing mount entry and edit its fs_type"
doesn't work because different fs_type would have incomptable
fs_options. For example, changing this:
  system  /system  ext4  ro,barrier=1  wait,logical,first_stage_mount
to this:
  system  /system  erofs  ro,barrier=1  wait,logical,first_stage_mount
doesn't work, because the erofs driver won't recognize the "barrier=1"
mount flag, since "barrier=1" is specific to ext4.

Bug: 235111004
Test: Boot erofs GSI.
Test: atest CtsFsMgrTestCases
Change-Id: I57132a55a089c7aae3e17c717ecd9dc1047fede8
2022-06-24 16:47:36 +08:00
Treehugger Robot
3be42ae576 Merge "Do not create /data/misc/zoneinfo." 2022-06-23 17:41:39 +00:00
Almaz Mingaleev
2f38c39169 Do not create /data/misc/zoneinfo.
APK time zone update mechanism used to store tzdata file there.
The feature is removed, no need to create that folder.

Bug: 148144561
Test: atest CtsBionicTestCases
Test: atest BionicTzdbConsistencyTest

Change-Id: I249f1d1b6c1a3f1a283d1ca43fcc93b10cbd910a
2022-06-23 17:21:39 +01:00
Treehugger Robot
d9a4563c53 Merge "Rewrite function declarators without prototypes" 2022-06-23 08:03:58 +00:00
Almaz Mingaleev
7f79ee42ee Merge "Remove TZUvA feature." 2022-06-23 07:47:26 +00:00
Steven Moreland
9d8af6af65 RefBase: disallow make_shared, make_unique
It's risky to mix multiple-ownership types. Taken from SharedRefBase.

Bug: 232557259
Test: N/A
Change-Id: Ic0dbd6d11e44fa9db87c4f9b1776d4989cbf9f56
2022-06-22 21:54:43 +00:00
Eric Biggers
e5a9a70437 Merge "Rename fscrypt_is_native() to IsFbeEnabled()" 2022-06-22 18:24:38 +00:00
Florian Mayer
ff5834a3e6 Merge "Promote HWASan tests to presubmit." 2022-06-22 17:55:41 +00:00
Yi Kong
d82c4e71c9 Rewrite function declarators without prototypes
This syntax is removed in C2x proposal N2841.

Test: presubmit
Change-Id: I3b067023327ed22dffda698c5f1accde60d95839
2022-06-22 16:59:22 +08:00
David Anderson
ebc8fe19f8 fastboot: Add a command to get current kernel logs.
This adds "fastboot getvar dmesg", usable only in userspace fastboot,
and only on unlocked devices. This is to debug flashing failures when
serial log access is not available.

Bug: 230269532
Test: fastboot getvar dmesg
Change-Id: Ic57881e362efe3f0687f41ab986d30b3a59dc4e4
2022-06-21 19:32:19 -07:00
Treehugger Robot
8874ad6378 Merge "Promote HWASan tests to presubmit." 2022-06-21 17:32:43 +00:00
Treehugger Robot
03d309beb2 Merge "Promote HWASan tests to presubmit." 2022-06-21 16:11:21 +00:00
Florian Mayer
556b4e0f80 Promote HWASan tests to presubmit.
They have been passing and do not show significant slowdown.

Change-Id: I8f6ac751515cffd4904e10d1ec473c3a1fff60ec
2022-06-17 22:41:33 +00:00
Florian Mayer
4effd2f901 Promote HWASan tests to presubmit.
They have been passing and do not show significant slowdown.

Change-Id: I3eba6ecb746160d75e9a2c9561bceb71a09117d3
2022-06-17 22:29:09 +00:00
Florian Mayer
db5b2c9fc0 Promote HWASan tests to presubmit.
They have been passing and do not show significant slowdown.

Change-Id: I64328118f0606c1b62a793fb8913eed5e5ba9578
2022-06-17 22:18:49 +00:00
Treehugger Robot
cef4850d9f Merge "DO NOT MERGE - Skip SQ3A.220605.009" 2022-06-17 14:29:00 +00:00
Xin Li
aed9c42ecb DO NOT MERGE - Skip SQ3A.220605.009
Bug: 236045730
Merged-In: Id47c567ad92ae4d9d7325a7a8589825a2ff4232b
Change-Id: I7c69dcd35188fb0be59cd1e37d07051c1a56375c
2022-06-16 22:50:49 -07:00
Treehugger Robot
e0f57d2883 Merge "[LSC] Add LOCAL_LICENSE_KINDS to system/core" 2022-06-16 19:14:41 +00:00
Bob Badour
453d3e4924 [LSC] Add LOCAL_LICENSE_KINDS to system/core
Added SPDX-license-identifier-Apache-2.0 to:
  debuggerd/test_permissive_mte/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all
Change-Id: Ic48cf8a972147eba8a955136be74204c013ca436
2022-06-16 10:01:01 -07:00
Akilesh Kailash
7e6cdc9f5e Merge "libsnapshot: Fix vts_libsnapshot_test for GRF with Vendor on S" 2022-06-15 23:24:47 +00:00
Akilesh Kailash
c898925104 libsnapshot: Fix vts_libsnapshot_test for GRF with Vendor on S
If the vendor partition is on S and system partition is on T,
certain tests in vts_libsnapshot_test used to fail. This is primarily
because of inconsistent check between daemon and vts test.

vts test checks the userspace.snapshots.enabled property which is true on T
but never checks if the underlying vendor partition is on S. Hence,
vts test will enable userspace snapshots. However, daemon checks
the vendor partition and disables userspace snapshots thereby
leading to inconsistency.

This is only a problem on vts tests. The underlying OTA on devices
works fine as we have the vendor partition check.

Bug: 231401995
Test: vts_libsnapshot_test on S vendor and T system
      vts_libsnapshot_test on T vendor and T system
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Iad4f299bd2e07c9c01f5fbee6a20e2f01bf1778a
2022-06-15 20:50:02 +00:00
Eric Biggers
893b1ae313 Rename fscrypt_is_native() to IsFbeEnabled()
Now that emulated FBE is no longer supported, there is no longer any
distinction between native FBE and emulated FBE.  There is just FBE.

Referring to FBE as "fscrypt" is also poor practice, as fscrypt (the
Linux kernel support for filesystem-level encryption) is just one part
of FBE, the Android feature.

Therefore, rename fscrypt_is_native() to IsFbeEnabled().

Bug: 232458753
Change-Id: Ib4dcdf799833b91d326b4b6f7120b0a3d4f4fc31
2022-06-15 18:52:39 +00:00
Florian Mayer
514c41c6e2 Merge "Add permissive MTE mode." 2022-06-15 16:58:25 +00:00
Treehugger Robot
79c6750b85 Merge "Chmod for existing nodes which mode not the same as ueventd.rc" am: a50d431830
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2063313

Change-Id: If9b135a575e643a5676b258513df8ec3f3484347
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-06-14 16:48:26 +00:00
Treehugger Robot
a50d431830 Merge "Chmod for existing nodes which mode not the same as ueventd.rc" 2022-06-14 14:14:15 +00:00
Treehugger Robot
b6ac0cc227 Merge "charge the permissions for the proc/bootconfig" am: 5683591ff9
Original change: https://android-review.googlesource.com/c/platform/system/core/+/2121492

Change-Id: Icf1aa5b773ec4eaa87648446305150f2d229d2f7
Signed-off-by: Automerger Merge Worker <android-build-automerger-merge-worker@system.gserviceaccount.com>
2022-06-13 20:55:41 +00:00
Treehugger Robot
5683591ff9 Merge "charge the permissions for the proc/bootconfig" 2022-06-13 20:17:11 +00:00