Commit graph

55069 commits

Author SHA1 Message Date
Treehugger Robot
e2e2b2d3d3 Merge "Fix error checking in append_string" 2019-12-20 00:38:25 +00:00
Nikita Ioffe
ee08c978d2 Merge "Now for real unmount bind-mounts on top of /data" 2019-12-20 00:01:28 +00:00
Nikita Ioffe
c9d8d2b7aa Merge "Don't proceed with userspace reboot if device doesn't support it" 2019-12-19 23:44:06 +00:00
Treehugger Robot
a275bc4b6a Merge "Merge libstatspush_compat and libstatssocket_q" 2019-12-19 23:31:55 +00:00
Treehugger Robot
0a31673879 Merge changes from topic "StatsEvent JNI"
* changes:
  Improve stats_event memory usage
  StatsEventCompat
  Remove libstats/Android.bp
  Make stats_event write function parameters const
  Create stats_buffer_writer
2019-12-19 23:31:48 +00:00
Ruchir Rastogi
3bbc240263 Fix error checking in append_string
Test: m -j
Change-Id: Ifa806d5037acb17f83cbf45712e9795d6f56152f
2019-12-19 14:57:10 -08:00
Nikita Ioffe
3d6a5fcb87 Now for real unmount bind-mounts on top of /data
Approach in
https://android-review.googlesource.com/c/platform/system/core/+/1191457
depended on ordering of entries. Instead fs_mgr now tries to unmount
everything with the same block device as /data until timeout is reached.
This is very similar to what is done in UmountPartitions in reboot.cpp,
but I'm a little bit reluctant to merge them together, since the later
unmount all read-write partitions, while here we are only interested in
/data.

Test: on both coral and blueline:
Test: adb shell /system/bin/vdc checkpoint startCheckpoint 1
Test: adb shell setprop sys.init.userdata_remount.force_umount_f2fs 1
Test: adb reboot userspace
Test: verified that userdata was remounted
Bug: 135984674
Change-Id: I929a058fa2f9639f0a2b73e9ea269cf1e07db889
2019-12-19 21:18:42 +00:00
Josh Gao
27fa358d41 Merge "adb: use poll instead of select in shell_service." 2019-12-19 20:22:13 +00:00
Songchun Fan
a1344fa6d7 Merge "[incremental] create /data/incremental in init.rc" 2019-12-19 18:14:06 +00:00
Treehugger Robot
0ba7842577 Merge "Create libstatssocket_q" 2019-12-19 17:36:08 +00:00
Peter Collingbourne
1d789eb210 Merge "Stop using weak symbols in libvndksupport." 2019-12-19 17:27:37 +00:00
Ruchir Rastogi
20516da9bc Merge libstatspush_compat and libstatssocket_q
This cleanup makes it clear that there are only two acceptable ways of
logging to the socket: either through libstatspush_compat or
libstatssocket (which will soon be renamed to libstatspush).

Test: m -j
Change-Id: I467b08bc300c2ef394b22422c6acf55a978a380a
Merged-In: I6a49b4995b56df462e1be901ea5e54acfe2e8bac
2019-12-18 17:48:28 -08:00
Ruchir Rastogi
79dd3eaa87 Improve stats_event memory usage
We now truncate the buffer to the appropriate length when clients call
stats_event_build().

Benchmarking tests indicate that truncating the buffer to the
appropriate length increases the cost clients pay to write to the socket
by 2%. This is negligible enough that I decided to truncate the buffer
for both pushed and pulled atoms in order to simplify the API.

Test: m libstatssocket
Test: bit libstatssocket_benchmark:*
Bug: 144126231
Change-Id: I35dec748ff87c0821d0d06779a406997e6e64966
Merged-In: Ife976bb383ecff8de5064730692a95e2a3a82c9d
2019-12-18 17:44:26 -08:00
Ruchir Rastogi
926fa88c35 StatsEventCompat
StatsEventCompat is a wrapper library that toggles between the old
logging scheme and the new logging scheme. It was designed particularly
for the DNS Resolver module and should not be used by others.

We will merge libstatspush_compat and libstatssocket_q in a future CL.

Test: m libstatspush_compat
Test: bit libstatspush_compat_test:* (passes on Q and R)
Bug: 145534143

Change-Id: Ib355031f3573101ea90bd8694861fbfc33b0c788
Merged-In: Idf35ccb6669798166475f08b2fbab40534b5db19
2019-12-18 17:21:27 -08:00
Muhammad Qureshi
a33f81bff5 Remove libstats/Android.bp
subdirs is not needed

Test: m -j
Change-Id: Idc034571eadae9c3135324af45fb4cd404ba8dd2
Merged-In: Ia7675fdf2e9a3a19bc52396ccbcc551556958803
2019-12-18 15:26:02 -08:00
Muhammad Qureshi
6d5a2fc44d Make stats_event write function parameters const
Bug: 143968790
Test: bit statsd_test:*
Change-Id: I413accb10cf3de4c5c49eb06a472168b1171039b
Merged-In: I2660cad5400a0e819c60b9caa706bb66ca2eef4f
2019-12-18 15:20:59 -08:00
Muhammad Qureshi
962cc23b9f Create stats_buffer_writer
Move iovec* construction and noteLogLoss from stats_event_list to
stats_buffer_writer.h.
Expose write_buffer_to_statsd that takes in a byte array and writes
it to the statsd socket. Currently exposed for StatsLog.write JNI call.

This change also allows getting rid of stats_event_list once all clients
have migrated to stats_event

Bug: 145619049
Test: m -j libstatssocket
Change-Id: I73f914985421fc1a4f094982c826bf364604b788
Merged-In: I0048e392c2f5039eb70dacf4e91a43d3f32e8749
2019-12-18 15:15:43 -08:00
Muhammad Qureshi
a3dcefc079 Create libstatssocket_q
Break up libstatssocket into libstatssocket_q and libstatssocket.
libstatssocket_q is for Q Mainline modules.

Bug: 145569088
Test: m -j
Test: bit statsd_test:*
Test: adb shell cmd stats print-logs && adb logcat "*:S statsd:*"
Change-Id: I2978dd544f4cb17714f3d14d5977cf6b46277ef9
Merged-In: I9d113b37640345ebad6aa269ab710db0d2179671
2019-12-18 15:11:53 -08:00
Treehugger Robot
1cdcc5f7e8 Merge "adb: fix use after free of atransport." 2019-12-18 22:39:48 +00:00
Treehugger Robot
d054eb7310 Merge "Simplified stats_event.c; exposed getter methods" 2019-12-18 22:27:27 +00:00
Treehugger Robot
a23ce83a3b Merge changes I15abdf13,I70304168,I52b0cb8e
* changes:
  General cleanup; add support for key-value pairs
  Move type ids to stats_event.h
  Set default timestamp to elapsedRealtimeNanos
2019-12-18 22:17:30 +00:00
Treehugger Robot
5158ab9ff2 Merge "First draft of native StatsEvent API" 2019-12-18 22:17:23 +00:00
Josh Gao
63da8e6272 adb: use poll instead of select in shell_service.
Bug: http://b/141955761
Test: test_device.py
Test: adbd_test
Change-Id: I8976304a1011e81e85f8d90d95d36ecd82834f5f
2019-12-18 14:02:57 -08:00
Ruchir Rastogi
a38d5a6fd6 Simplified stats_event.c; exposed getter methods
Simplifications/changes:
(1) We remove the bufPos field and track the end of the buffer using
    only the size field.
(2) We rename put_<type>() functions to append_<type>() to signify that the
    value will be placed at the end of the buffer. We also move the
    increment of event->size to within the append_<type>() functions;
    this improves readability of the write_<type>() functions.
(3) We immediately write the timestamp and atom id to the buffer in
    order to simplify stats_event_build().
(4) We never check for null pointers; checking for null pointers
    delays errors and obfuscates the root problem.
(5) We change the the annotationId and numPairs parameters to be
    uint8_t's. This helps signify to clients that these values must fit
    in 1 byte.
(6) Clients no longer have to pass in the length of strings. Instead, we
    expect them to pass in null-terminated strings, and we will
    calculate the length outselves using strnlen.

Test: m -j libstatssocket
Change-Id: I6d192768876a23d7016173bcdaf59f8b7e92b182
Merged-In: I0173f8bc76ef25118379dce5d2481f5f7a9b7519
2019-12-18 13:17:36 -08:00
Ruchir Rastogi
3822e61d7b General cleanup; add support for key-value pairs
Encoding of key-value pairs follows strategy described in
go/statsd-socket-doc.

Fixed bug where timestamp and atom id were not contributing to the
number of fields in the buffer.

Note: Similar to ag/9603344, which was abandoned because of git rebasing
issues.

Test: m -j libstatssocket compiles
Bug: 143079361
Change-Id: I15abdf13ad8debf83ae1f6c372676db4fe390289
Merged-In: I8d23ddf2dbfee518ad970894ac09ba3ad5b5b4ad
2019-12-18 13:11:28 -08:00
Ruchir Rastogi
a3bc1af1b8 Move type ids to stats_event.h
This allows the parsing code within statsd to access the type ids.
Also, we move stats_event.h to the include directory for consistency.

Test: m -j libstatssocket
Change-Id: I7030416872677f3a523c2baf7bf6bcb4ca0ae6a4
Merged-In: I1db7fc9e9d0efd2225612353a8fc49bb13220ff0
2019-12-18 11:58:42 -08:00
Ruchir Rastogi
f4d23fc353 Set default timestamp to elapsedRealtimeNanos
Test: m -j libstatssocket compiles
Change-Id: I52b0cb8e1e916413db3dc0fe1b9a7e922833bdf7
Merged-In: I62f4c6bb14c3914124a6af25ff46fb0a980204e7
2019-12-18 11:53:38 -08:00
Ruchir Rastogi
bb0da94f8d First draft of native StatsEvent API
Test: m -j libstatssocket compiles
Change-Id: I874968d4f77a5990764100123794b923847e5eb5
Merged-In: I0da9740d29749d79d14be39e5949b98c58df704d
2019-12-18 11:33:22 -08:00
Nikita Ioffe
9f473c0579 Don't proceed with userspace reboot if device doesn't support it
Test: set property to false and adb reboot userspace
Bug: 135984674
Change-Id: I11e55243df002dd54e533a6b9fc529866597d4dd
2019-12-18 18:06:47 +00:00
Songchun Fan
284962431b [incremental] create /data/incremental in init.rc
We use /data/incremental to keep all the directories that are mounted on
the Incremental File System.

Since system_server does not have permission to dynamically create dirs
directly under /data, we create /data/incremental in init.rc.

Test: boots
BUG: 136132412
Change-Id: Ic90cc8f652672a8d4459c0cd38db9c0872217af4
2019-12-18 09:26:51 -08:00
Elliott Hughes
264a37d12f Merge "Fix SEGV in libziparchive with malformed zip file." 2019-12-18 16:14:54 +00:00
David Anderson
43ad6362da Merge "liblp: Add a VIRTUAL_AB_DEVICE flag for LpMetadataHeader." 2019-12-18 15:42:44 +00:00
David Anderson
42c86c763c Merge "libfiemap: Fix fallocate() usage." 2019-12-18 15:37:10 +00:00
Treehugger Robot
90e9217aca Merge "Enable "localfilesystem" UNIX domain socket for ADB." 2019-12-18 05:34:30 +00:00
Peter Collingbourne
683192270f Stop using weak symbols in libvndksupport.
These weak symbols are no longer necessary because we no longer
need to statically link libvndksupport.

Furthermore, these symbols are not really optional when libvndksupport is
dynamically linked. When the library is linked with --as-needed the linker
may drop the reference to libdl_android.so, which leads to a failure to boot.

Bug: 146456667
Change-Id: I4d2290d8d0206e0d89665d4d872a7c53b3e1531b
2019-12-17 17:52:05 -08:00
David Anderson
71b287bfd7 libfiemap: Fix fallocate() usage.
This fixes two bugs in how we use fallocate(). First, using ZERO_RANGE
is a mechanism to allocate holes, not blocks. Instead we should be
passing no flags.

The reason this code worked is because of our explicit writes to the
file, which is intended to defeat lazy block allocation. This is not
necessary for F2FS with proper file pinning support, so we now skip this
when possible.

Together, these fixes improve the speed of "adb remount" when backed
by libfiemap.

Bug: 134949511
Test: adb remount when backed by /data
Change-Id: I746053d183e0c83d56979d9c3fa6d96b40b5e478
2019-12-17 15:04:08 -08:00
Yifan Hong
18c6248ffe Merge changes from topic "wait_for_merge"
* changes:
  libsnapshot: add WaitForMerge
  libsnapshot: SnapshotUpdateTest::AddOperation
  libsnapshot: add GetCurrentSlot
2019-12-17 23:01:59 +00:00
Treehugger Robot
bc58945cd1 Merge "Revert "Have /storage always point to sdcardfs by default."" 2019-12-17 17:38:47 +00:00
David Anderson
1526a46e92 Merge "fs_mgr: Move libfiemap back to fs_mgr from system/gsid." 2019-12-17 17:32:09 +00:00
Elliott Hughes
fba2a1a1ec Fix SEGV in libziparchive with malformed zip file.
d77c99ebc3 changed MappedFile to return a
bogus zero-length mapping on failure rather than nullptr. None of the
calling code was changed, though, and it seems like doing so would be a
bad idea. Revert that part of the change.

Add missing tests, and tidy up some of the logging. Also remove
single-use or obfuscatory constants from the tests.

The new "empty.zip" was created by using zip(1) to create a zip file
with one entry, then using `zip -d` to remove it.

The new "zero-size-cd.zip" was created by using zip(1) to create a zip
file containing a single empty file, and then hex editing the two byte
"size of the central directory" field in the "end of central directory
record" structure at the end of the file. (This is equivalent to, but
much smaller than, the example zip file provided by the bug reporter.)

Bug: http://b/145925341
Test: treehugger
Change-Id: Iff64673bce7dae886ccbc9dd6c2bbe18de19f9d2
2019-12-17 08:39:09 -08:00
Martijn Coenen
0dbb2a75f2 Revert "Have /storage always point to sdcardfs by default."
This reverts commit 5d53bfce2d.

Reason for revert: for consistency, we've decided that for now, all code will have the same view of /storage: FUSE. Will address the TODO here later.

Change-Id: Ia75e23c91fb098f6309c160de2889f06507c3717
2019-12-17 13:45:10 +00:00
Nikita Ioffe
e4b931146d Merge "Unmount bind-mounts on top of /data before unmounting /data" 2019-12-17 12:06:59 +00:00
Ed Chen
701abc3f14 Merge "Revert "remount: Remove the "backing" parameter to fs_mgr_overlayfs_setup."" 2019-12-17 09:18:53 +00:00
Ed Chen
1981401909 Revert "remount: Remove the "backing" parameter to fs_mgr_overlayfs_setup."
This reverts commit b20aa434a3.

Bug: 146403375
Reason for revert: broken build 6078429 on git_master on walleye-user

Change-Id: Idc3b1c0ad5228d63298ce84811624b23d0cea434
2019-12-17 08:53:13 +00:00
David Anderson
905ecdd374 Merge "remount: Remove the "backing" parameter to fs_mgr_overlayfs_setup." 2019-12-17 06:05:43 +00:00
David Anderson
4fe398018b fs_mgr: Move libfiemap back to fs_mgr from system/gsid.
Bug: 134949511
Test: builds
Change-Id: I07943acce5d4671975afef6d8da1bdbeef43c39d
2019-12-16 20:10:26 -08:00
Nikita Ioffe
0dda423dd0 Unmount bind-mounts on top of /data before unmounting /data
Test: adb shell setprop sys.init.userdata_remount.force_umount_f2fs 1
Test: adb shell /system/bin/vdc checkpoint startCheckpoint 1
Test: adb reboot userspace
Bug: 135984674
Bug: 143970043
Change-Id: Icd7f3ad1c42d9b21bf4eacf1dec60355c9883254
2019-12-16 23:22:55 +00:00
David Anderson
4ba68b6451 liblp: Add a VIRTUAL_AB_DEVICE flag for LpMetadataHeader.
Bug: 134949511
Test: liblp_test gtests
Change-Id: I8b376d0980e1372086ed15b980209f3726ef651a
2019-12-16 22:20:40 +00:00
Treehugger Robot
48d3b1e112 Merge "fastboot: Flashall does proper snapshot cancel" 2019-12-16 21:44:39 +00:00
Treehugger Robot
cd652ea3dc Merge "Rename unzip.cpp to ziptool.cpp." 2019-12-16 18:37:49 +00:00