Commit graph

86782 commits

Author SHA1 Message Date
Treehugger Robot
8097002e09 Merge "Check if ALWAYS_INLINE is already defined" into main 2024-02-14 20:40:30 +00:00
Serdar Kocdemir
593b19b5ce Check if ALWAYS_INLINE is already defined
Fixes build errors when using callstack library in gfxstream
project, where the definition is already made.

Test: build
Change-Id: I80d4038160725b5dc70f9af93d338aca66a2f0d5
2024-02-14 17:52:19 +00:00
Florian Mayer
75800c9c3a Merge "Read data set by android_add_crash_detail into tombstone." into main 2024-02-14 17:06:07 +00:00
Treehugger Robot
619042f207 Merge "first_stage_console: Address compiler warning" into main 2024-02-14 15:34:45 +00:00
Snehal Koukuntla
66dc1bde80 Merge "Sync tipc.h with one from trusty" into main 2024-02-14 09:07:24 +00:00
Florian Mayer
5fa6663458 Read data set by android_add_crash_detail into tombstone.
Bug: 155462331
Bug: 309446525
Change-Id: I6d01aafca48e0e5e8cbd5ae87add6aec0c429503
2024-02-13 18:13:22 -08:00
Vilas Bhat
89fa277dcb Merge "ashmem: Remove hardcoded PAGE_SIZE usage in ashmem_test" into main 2024-02-14 00:51:11 +00:00
Treehugger Robot
222ba3e8bd Merge "Change debuggerd client param from pid to tid" into main 2024-02-14 00:13:33 +00:00
Vilas Bhat
f791b99628 ashmem: Remove hardcoded PAGE_SIZE usage in ashmem_test
Bug: 323061793
Test: atest libcutils_test
Change-Id: I8defd5225859b52a1b96a638d418188b328a23bc
2024-02-13 20:50:00 +00:00
Snehal
f425428c2d Sync tipc.h with one from trusty
Bug: 325069766

Test: Android boot and crasher test

Change-Id: I36244cfb698a81be79c204eb019e1a59b90d2e6c
2024-02-13 18:10:00 +00:00
Pierre-Clément Tosi
914ccc5664 first_stage_console: Address compiler warning
Re-order the fields in the declaration to address the following

  ISO C++ requires field designators to be specified in declaration order;
  field 'sigaction::(anonymous union at bionic/libc/include/bits/signal_types.h:76:20)'
  will be initialized after field 'sa_flags' [-Wreorder-init-list]

Test: TH
Change-Id: I3f1d4739f1677aad50d4f07361aa264da43bdd13
2024-02-12 18:57:16 +00:00
Ryan Prichard
18744c17c1 Merge "libsnapshot: replace non-character basic_string[_view]<T>" into main 2024-02-09 20:00:06 +00:00
Treehugger Robot
0bd010d9ae Merge "Remove obsolete __STDC_LIMIT_MACROS references." into main 2024-02-09 19:11:47 +00:00
Elliott Hughes
29cd07132d Remove obsolete __STDC_LIMIT_MACROS references.
None of our libcs need this any more.

Test: treehugger
Change-Id: I8a942ad6b3860633a6d722bd237a7c75482499eb
2024-02-09 16:40:02 +00:00
Treehugger Robot
22e1c5dcb0 Merge "Replace lazy_static with once_cell" into main 2024-02-09 15:41:46 +00:00
Ryan Prichard
abb472c238 libsnapshot: replace non-character basic_string[_view]<T>
In newer versions of libc++, std::char_traits<T> is no longer defined
for non-character types, and a result, std::basic_string<T> and
std::basic_string_view<T> are also no longer defined for non-character
types. See
https://discourse.llvm.org/t/deprecating-std-string-t-for-non-character-t/66779.

Replace them with std::vector<T> and std::span<const T>.

Bug: 175635923
Test: m MODULES-IN-system-core-fs_mgr
Test: /data/nativetest64/cow_api_test/cow_api_test
Change-Id: Ife2e87833ced43ff24e5765998cb6993e4f9b4c0
2024-02-08 23:20:10 -08:00
Treehugger Robot
ff239bb0fc Merge "Add test mapping" into main 2024-02-08 18:30:25 +00:00
T.J. Mercier
c902df07b0 Add test mapping
Bug: 315371696
Change-Id: If8448e703cc5accde11bcdccf7585ef6779f59b0
2024-02-08 16:57:43 +00:00
Treehugger Robot
2d686d7b71 Merge "fastboot: Use clock_gettime(CLOCK_MONOTONIC) in now()" into main 2024-02-08 16:09:44 +00:00
Treehugger Robot
4e4d918689 Merge "Remove unused addKernelPanicSubReason() return value." into main 2024-02-08 16:09:04 +00:00
Neill Kapron
2b2fcaea46 Merge "healthd: BatteryMonitor: doLogValues: reduce log level to debug" into main 2024-02-08 03:24:16 +00:00
Treehugger Robot
241046adaa Merge "fastboot: Remove '\n' from interface name on linux" into main 2024-02-08 03:11:53 +00:00
Peter Collingbourne
cc1bf34ba6 fastboot: Use clock_gettime(CLOCK_MONOTONIC) in now()
If the system time changes during the execution of fastboot we might
see some strange output such as:

Sending sparse 'super' 4/20 (254972 KB)            OKAY [-516.263s]

Fix it by changing now() to use clock_gettime(CLOCK_MONOTONIC).
I confirmed that all callers of now() are using it for relative time
and not time since the epoch.

Change-Id: Ic3e9442c2ab21dfb076bfed88915085a183754b0
2024-02-07 16:55:49 -08:00
Matt Schulte
28f4807dd3 fastboot: Remove '\n' from interface name on linux
If the device provides an interface string, we should remove the newline
from the read file that is added from by the linux kernel.

Bug: 324320178
Test: Ran the same command with my local changes vs not

```
bash$ ./fastboot devices -l
5f42ad5ad259c90cf14ea222791b6aaa fastboot usb:7-3

bash$ fastboot devices -l
5f42ad5ad259c90cf14ea222791b6aaa fastboot
 usb:7-3
```

Change-Id: Ida3316fdba8e35f0c66784f83455a4d82e90ba1c
2024-02-07 16:24:26 -08:00
Neill Kapron
aeccbb8f3a healthd: BatteryMonitor: doLogValues: reduce log level to debug
This is log spam and fills the kernel log. Reduce log level from warning
to debug.

Test: manual
Bug: 36785118
Change-Id: I70fc3a0d796cb1a984705236fe7527e266a69a03
Signed-off-by: Neill Kapron <nkapron@google.com>
2024-02-07 23:37:50 +00:00
Elliott Hughes
1c5911fade Remove unused addKernelPanicSubReason() return value.
The only caller that was looking at the return value just stopped.

Test: treehugger
Change-Id: Icf2508e8e1ee4969918eadeb580f70236ba53496
2024-02-07 22:22:13 +00:00
Elliott Hughes
c72c9a6d3c Merge "We don't need __STDC_FORMAT_MACROS any more." into main 2024-02-07 21:56:38 +00:00
Elliott Hughes
6225d6f3eb We don't need __STDC_FORMAT_MACROS any more.
Test: treehugger
Change-Id: Iada8b959f61bfa55b61d2ed4b345801ed9cea0c6
2024-02-07 18:27:47 +00:00
Treehugger Robot
0c78cf96a1 Merge "bootstat: Remove long key press check" into main 2024-02-07 18:23:30 +00:00
Treehugger Robot
277533b7a3 Merge "fastboot: Use asynchronous operations to send data" into main 2024-02-07 03:42:00 +00:00
Zheng Pan
fa544b551b bootstat: Remove long key press check
Current check is kind of device specfic and causes false positive.

Bug: 320829712
Test: Trigger long key press and check boot reason
Change-Id: I5b2b8307f16eecd05513bfa51494da174fc839bd
2024-02-06 17:39:27 -08:00
Peter Collingbourne
8635c016db fastboot: Use asynchronous operations to send data
The fastboot command currently uses USBDEVFS_BULK to transfer data
(including image data) to the target. On the kernel side it looks
like this:

1. Allocate a contiguous memory region and copy the user data into
   the memory region (which may involve accessing storage).
2. Instruct the driver to start a DMA operation.
3. Wait for the DMA to finish.

This is suboptimal because it misses out on a pipelining
opportunity. We could be doing 3 for the current operation in parallel
with 1 for the next operation, so that the next DMA is ready to go
as soon as the current DMA finishes.

The kernel supports asynchronous operations on usbdevfs file
descriptors (USBDEVFS_SUBMITURB and USBDEVFS_REAPURB), so we can
implement this like so:

1. Submit URB 0
2. Submit URB 1
3. Wait for URB 0
4. Submit URB 2
5. Wait for URB 1
and so on.

That is what this CL implements. On my machine it increases transfer
speed from 125 MB/s to 160 MB/s using a USB 3.0 connection to the
target (Pixel 8).

Bug: 324107907
Change-Id: I20db7ea14af85db48f6494091c8279ef7a21033d
2024-02-06 13:10:27 -08:00
Jeffrey Vander Stoep
40b6a8f0c6 Merge "Replace use of deprecated logging functions" into main 2024-02-06 16:09:30 +00:00
Jeff Vander Stoep
56aedd5551 Replace use of deprecated logging functions
This is needed to upgrade the android_logger crate from 0.12.0
to 0.13.3.

with_max_level provides the same functionality as with_min_level.
The renaming is admittedly confusing, but the new name is accurate
and it makes sense that they deprecated and then removed the
previously poorly named with_min_level.

See crate documentation [1] and code [2].

[1]: https://docs.rs/android_logger/0.12.0/android_logger/struct.Config.html#method.with_min_level
[2]: https://docs.rs/android_logger/0.12.0/src/android_logger/lib.rs.html#227

Bug: 322718401
Test: build and run CF with the change.
Test: m aosp_cf_x86_64_phone
Change-Id: Ib4fbd486267d30e74e886139846950b066848d43
2024-02-06 12:32:36 +01:00
David Dai
161b80bd02 Merge "Add AID for Virtual Machines" into main 2024-02-06 01:21:39 +00:00
Treehugger Robot
eadf6b4bda Merge "remount: Auto remount option (-R) also tries to gain root" into main 2024-02-05 23:47:54 +00:00
David Dai
851c9c825f Add AID for Virtual Machines
Add a new AID for Virtual Machines so we can grant
capabilities such as CAP_SYS_NICE.

Bug: 322197421
Test: Build and boots, and verified capabilities

Change-Id: Ie893ba8ed6956a554bccfbd00e4e6fe9212ea77d
Signed-off-by: David Dai <davidai@google.com>
2024-02-05 23:38:44 +00:00
Kenn Frankel
7833f27c25 Merge "Update libstatssocket_lazy to support StringArray" into main 2024-02-05 19:16:33 +00:00
Treehugger Robot
749e2e8e2d Merge "Restore I/O priority support" into main 2024-02-02 01:13:16 +00:00
Kenn Frankel
374c43e9b7 Update libstatssocket_lazy to support StringArray
Bug: 281162849
Test: LibstatssocketLazyTest

Change-Id: I917589e92db045f96daa9b191fadac50b01704c8
2024-02-01 11:49:51 -08:00
Bart Van Assche
0cf102b703 Restore I/O priority support
CL aosp/2929791 removed I/O priority support to prepare for a clean
revert of the CL that migrates the blkio controller from the v1 to the
v2 cgroup hierarchy. Since there was no other reason to revert the I/O
priority CL, restore I/O priority support.

Bug: 186902601
Change-Id: I1a4053140ab55973878bfeacfb546da3c601a895
Signed-off-by: Bart Van Assche <bvanassche@google.com>
2024-02-01 10:52:13 -08:00
Akilesh Kailash
942b213628 Merge changes from topic "ota-block-size-compression" into main
* changes:
  snapuserd: Add I/O path support for variable block size
  libsnapshot_cow: Support multi-block compression
2024-02-01 18:13:02 +00:00
Devin Moore
b56e4743f6 Change debuggerd client param from pid to tid
Change the header param to be `tid` and remove a local `pid` variable
to use `tid` instead.

Test: m
Bug: 316970771
Change-Id: I53c13081d72f46446ac6e85df77a160ef4f50c05
2024-01-31 23:44:01 +00:00
Akilesh Kailash
3ea911bc06 snapuserd: Add I/O path support for variable block size
The flow of I/O path is as follows:

1: When there is a I/O request for a given sector, we first
check the in-memory COW operation mapping for that sector.

2: If the mapping of sector to COW operation is found, then the
existing I/O path will work seamlessly. Even if the COW operation
encodes multiple blocks, we will discard the remaining data.

3: If the mapping of sector to COW operation is not found:
    a: Find the previous COW operation as the vector has sorted sectors.

    b: If the previous COW operation is a REPLACE op:
        i: Check if the current sector is encoded in the previous COW
	operations compressed block.

	ii: If the sector falls within the range of compressed blocks,
	retrieve the block offset.

	iii: De-compress the COW operation based on the compression
	factor.

	iv: memcpy the data based on the block offset.

	v: cache the COW operation pointer as subsequent I/O requests
	are sequential and can just be a memcpy at the correct offset.
    c: If the previous COW operation is not a REPLACE op or if the
       requested sector does not fall within the compression factor
       of the previous COW operation, then fallback and read the data
       from base device.

Snapshot-merge:

During merge of REPLACE ops, read the entire op in one shot, de-compress
multiple blocks and write all the blocks in one shot.

Performance:

go/variable-block-vabc-perf covers detail performance runs
on Pixel 6 for full and incremental OTA.

Bug: 319309466

Test: snapuserd_test covers all the I/O path with various block sizes.
About 252 cases with all combinations and tunables.

[==========] 252 tests from 4 test suites ran. (702565 ms total)
[  PASSED  ] 252 tests.

On Pixel 6:

=======================================
COW Writer V3:

for i in full, incremental OTA
   for j in 4k, 16k, 32k, 64k, 128, 256k
      for k in lz4, zstd, gz
	 install OTA, reboot, verify merge
=======================================
COW Writer V2:

for i in full, incremental OTA
  for j in 4k
    for k in lz4, zstd, gz
      install OTA, reboot, verity merge

=====================================

Change-Id: I4c3b5c3efa0d09677568b4396cc53db0e74e7c99
Signed-off-by: Akilesh Kailash <akailash@google.com>
2024-01-31 13:28:45 -08:00
Akilesh Kailash
59fa486703 libsnapshot_cow: Support multi-block compression
This patch supports compression for bigger block size.

3 bits [57-59] in the COW Operation "source_info_" field is used to store
the compression factor. Supported compression factors are power of 2
viz: 4k, 8k, 16k, 32k, 64k, 128k, 256k.

Only REPLACE operations will have the bigger block size support for now.
This can be extended to other operations later.

The write path in EmitBlocks() has the core logic wherein consecutive
sequence of REPLACE ops are compressed based on the compression factor
settings. Thus, for a 64k compression factor, there will be just one
COW operation which encodes all the 16 operation and the entire 64k
block is compressed in one shot.

NOTE: There is no read I/O path support in this patch. Subsequent patch
will have the read support.

Performance data (with read I/O path support in subsequent patch):

go/variable-block-vabc-perf covers detail performance runs
on Pixel 6 for full and incremental OTA.

TL;DR:

Performance of a full OTA (All numbers are compared against 4k block
size)
=======================================
Snapshot-size:

~10-11% decrease in snapshot-size (disk-space) for zstd with 256k block
size.

~8% decrease in snapshot-size (disk-space) for lz4

Install time:

~13% decrease in OTA install time for zstd with 256k block size.

Snapshot-merge:

~50% decrease in snapshot-merge time with 256k block size for zstd

Post OTA boot-time:

~10.5 decrease in boot time for 64k block size for zstd

In-memory footprint for COW operations:

~80% decrease in memory footprint for 256k block size. (58MB -> 9.2MB)

============================================

For more improvements, further tuning of zstd/lz4 is
required primariy the compression levels, zstd compression window,
performance of gz with compression levels.

Bug: 319309466

Test: cow_api test covering all the supported block sizes for v3 writer.

On Pixel 6:

=======================================
COW Writer V3:

for OTA in full, incremental OTA
   for block_size in 4k, 16k, 32k, 64k, 128k, 256k
       for compression_algo in lz4, zstd, gz, none
          install OTA, reboot, verify merge
=======================================
COW Writer V2:

for OTA in full, incremental OTA
   for block_size in 4k
      for compression_algo in lz4, zstd, gz, none
          install OTA, reboot, verity merge

=====================================
Change-Id: I96201f1609582aa9d44d8085852e284b0c4a426d
Signed-off-by: Akilesh Kailash <akailash@google.com>
2024-01-31 12:52:31 -08:00
Akilesh Kailash
13cb6f7b4b Merge "libsnapshot: set header max_compression" into main 2024-01-31 20:42:58 +00:00
Daniel Zheng
bcce91603b libsnapshot: set header max_compression
Intermediate CL needed before variable block size can land. Since v3 is
enabled on cuttlefish, the base build needs to write the
compression_factor in order for reader to properly parse. Otherwise
we'll fail OTA test

Test: th
Change-Id: Ia353aae8e668858851073f09308909ae70d7854e
2024-01-31 17:35:00 +00:00
David Drysdale
700ff706f1 Merge "Secretkeeper: require SECRETKEEPER_ENABLED=true" into main 2024-01-31 07:13:24 +00:00
Jiyong Park
0edd5b860a Merge "Drop CtsInitTestCases from cts" into main 2024-01-31 00:58:50 +00:00
Akilesh Kailash
14258369ef Merge "libsnapshot: update type" into main 2024-01-30 19:23:49 +00:00