android_system_core/logd
Tom Cherry ec39a3f6df logd: record and replay log messages
To profile different log buffer types and configuration, this change
adds the ability to record log messages and adds a tool that will
replay those log messages through different log buffer implementations
and collect stats about the execution.

Test: log messages replay correctly
Change-Id: I0dc6c545b782fa7732e325dde109c496b137d0dd
2020-07-17 15:08:21 -07:00
..
fuzz logd: optionally track the full size of log buffers 2020-07-01 14:35:33 -07:00
.clang-format logd: use .clang-format-4 like the rest of system/core 2019-01-16 15:20:15 -08:00
Android.bp logd: record and replay log messages 2020-07-17 15:08:21 -07:00
AndroidTest.xml logd: add LogBufferTest.cpp 2020-05-18 15:51:08 -07:00
auditctl.cpp introduce auditctl and use it to configure SELinux throttling 2019-04-09 13:19:08 -07:00
ChattyLogBuffer.cpp logging: Use more inclusive language. 2020-06-16 14:31:31 -07:00
ChattyLogBuffer.h logging: Use more inclusive language. 2020-06-16 14:31:31 -07:00
ChattyLogBufferTest.cpp logd: move leading_dropped logic into FlushTo() 2020-06-02 13:26:48 -07:00
CommandListener.cpp logging: Use more inclusive language. 2020-06-16 14:31:31 -07:00
CommandListener.h logging: Use more inclusive language. 2020-06-16 14:31:31 -07:00
CompressionEngine.cpp logd: replace std::vector<uint8_t> in SerializedLogChunk 2020-06-24 16:19:28 -07:00
CompressionEngine.h logd: replace std::vector<uint8_t> in SerializedLogChunk 2020-06-24 16:19:28 -07:00
event.logtags
libaudit.cpp logd: make liblogd just the core logd functionality 2020-05-18 13:03:48 -07:00
libaudit.h logd: make liblogd just the core logd functionality 2020-05-18 13:03:48 -07:00
LogAudit.cpp logd: make LogBuffer an interface 2020-05-12 15:39:19 -07:00
LogAudit.h logd: refactor LastLogTimes a bit 2020-05-12 15:39:11 -07:00
LogBuffer.h logd: move leading_dropped logic into FlushTo() 2020-06-02 13:26:48 -07:00
LogBufferElement.cpp logd: optionally track the full size of log buffers 2020-07-01 14:35:33 -07:00
LogBufferElement.h logd: fix various clang-tidy issues 2020-06-17 11:40:55 -07:00
LogBufferTest.cpp logd: add a SerializedLogBuffer suitable for compression 2020-06-12 14:35:30 -07:00
LogBufferTest.h logd: optionally track the full size of log buffers 2020-07-01 14:35:33 -07:00
logd.rc logd: don't drop user/groups/capabilities/priority in drop_privs() 2019-06-07 15:52:27 +00:00
logd_test.cpp logd: add LogBufferTest.cpp 2020-05-18 15:51:08 -07:00
LogKlog.cpp logging: Use more inclusive language. 2020-06-16 14:31:31 -07:00
LogKlog.h logd: remove SocketClient from LogBuffer and LogBufferElement 2020-05-14 19:53:45 -07:00
LogListener.cpp logd: remove class LogCommand, rename LogCommand.cpp to LogPermissions.cpp 2020-05-15 11:58:47 -07:00
LogListener.h logd: fix various clang-tidy issues 2020-06-17 11:40:55 -07:00
LogPermissions.cpp logd: fix various clang-tidy issues 2020-06-17 11:40:55 -07:00
LogPermissions.h logd: remove class LogCommand, rename LogCommand.cpp to LogPermissions.cpp 2020-05-15 11:58:47 -07:00
LogReader.cpp logd: fix various clang-tidy issues 2020-06-17 11:40:55 -07:00
LogReader.h logd: remove SocketClient from LogBuffer and LogBufferElement 2020-05-14 19:53:45 -07:00
LogReaderList.cpp logd: create FlushToState class 2020-06-01 14:45:02 -07:00
LogReaderList.h logd: create FlushToState class 2020-06-01 14:45:02 -07:00
LogReaderThread.cpp logd: fix various clang-tidy issues 2020-06-17 11:40:55 -07:00
LogReaderThread.h logd: add a SerializedLogBuffer suitable for compression 2020-06-12 14:35:30 -07:00
LogStatistics.cpp logd: record and replay log messages 2020-07-17 15:08:21 -07:00
LogStatistics.h logd: record and replay log messages 2020-07-17 15:08:21 -07:00
logtagd.rc Relax permissions on /data/misc/logd 2020-01-21 18:39:19 -08:00
LogTags.cpp logd: add fuzzer for SerializedLogBuffer and other improvements 2020-06-23 11:37:49 -07:00
LogTags.h logd: create SimpleLogBuffer and implement ChattyLogBuffer in terms of it 2020-05-21 14:23:45 -07:00
LogUtils.h logd: use libbase logging 2020-06-03 16:29:48 -07:00
LogWriter.h logd: drop can_read_security_logs 2020-06-01 14:45:02 -07:00
main.cpp logd: allow opting into simple or serialized log buffers 2020-07-14 11:52:06 -07:00
OWNERS Add OWNERS. 2017-12-07 13:30:03 -08:00
PruneList.cpp logging: Use more inclusive language. 2020-06-16 14:31:31 -07:00
PruneList.h logging: Use more inclusive language. 2020-06-16 14:31:31 -07:00
README.auditd
README.property logd: allow opting into simple or serialized log buffers 2020-07-14 11:52:06 -07:00
README.replay.md logd: record and replay log messages 2020-07-17 15:08:21 -07:00
RecordedLogMessage.h logd: record and replay log messages 2020-07-17 15:08:21 -07:00
RecordingLogBuffer.cpp logd: record and replay log messages 2020-07-17 15:08:21 -07:00
RecordingLogBuffer.h logd: record and replay log messages 2020-07-17 15:08:21 -07:00
ReplayMessages.cpp logd: record and replay log messages 2020-07-17 15:08:21 -07:00
rwlock.h logd: fix various clang-tidy issues 2020-06-17 11:40:55 -07:00
SerializedData.h logd: replace std::vector<uint8_t> in SerializedLogChunk 2020-06-24 16:19:28 -07:00
SerializedFlushToState.cpp logd: always compress SerializedLogChunk in FinishWriting() 2020-07-16 20:46:14 -07:00
SerializedFlushToState.h logd: fix use after resize of contents_ vector 2020-06-24 15:31:46 -07:00
SerializedFlushToStateTest.cpp logd: fix use after resize of contents_ vector 2020-06-24 15:31:46 -07:00
SerializedLogBuffer.cpp logd: always compress SerializedLogChunk in FinishWriting() 2020-07-16 20:46:14 -07:00
SerializedLogBuffer.h logd: don't use a thread for deleting log chunks 2020-07-14 18:50:45 +00:00
SerializedLogChunk.cpp logd: always compress SerializedLogChunk in FinishWriting() 2020-07-16 20:46:14 -07:00
SerializedLogChunk.h logd: always compress SerializedLogChunk in FinishWriting() 2020-07-16 20:46:14 -07:00
SerializedLogChunkTest.cpp logd: always compress SerializedLogChunk in FinishWriting() 2020-07-16 20:46:14 -07:00
SerializedLogEntry.h logd: optionally track the full size of log buffers 2020-07-01 14:35:33 -07:00
SimpleLogBuffer.cpp logd: simplify Clear() + Prune() logic 2020-06-11 07:59:01 -07:00
SimpleLogBuffer.h logd: fix various clang-tidy issues 2020-06-17 11:40:55 -07:00

logd can record and replay log messages for offline analysis.

Recording Messages

logd has a RecordingLogBuffer buffer that records messages to /data/misc/logd/recorded-messages. It stores messages in memory until that file is accessible, in order to capture all messages since the beginning of boot. It is only meant for logging developers to use and must be manually enabled in by adding RecordingLogBuffer.cpp to Android.bp and setting log_buffer = new SimpleLogBuffer(&reader_list, &log_tags, &log_statistics); in main.cpp.

Recording messages may delay the Log() function from completing and it is highly recommended to make the logd socket in liblog blocking, by removing SOCK_NONBLOCK from the socket() call in liblog/logd_writer.cpp.

Replaying Messages

Recorded messages can be replayed offline with the replay_messages tool. It runs on host and device and supports the following options:

  1. interesting - this prints 'interesting' statistics for each of the log buffer types (simple, chatty, serialized). The statistics are:
    1. Log Entry Count
    2. Size (the uncompressed size of the log messages in bytes)
    3. Overhead (the total cost of the log messages in memory in bytes)
    4. Range (the range of time that the logs cover in seconds)
  2. memory_usage BUFFER_TYPE - this prints the memory usage (sum of private dirty pages of the replay_messages process). Note that the input file is mmap()'ed as RO/Shared so it does not appear in these dirty pages, and a baseline is taken before allocating the log buffers, so only their contributions are measured. The tool outputs the memory usage every 100,000 messages.
  3. latency BUFFER_TYPE - this prints statistics of the latency of the Log() function for the given buffer type. It specifically prints the 1st, 2nd, and 3rd quartiles; the 95th, 99th, and 99.99th percentiles; and the maximum latency.
  4. print_logs BUFFER_TYPE [buffers] [print_point] - this prints the logs as processed by the given buffer_type from the buffers specified by buffers starting after the number of logs specified by print_point have been logged. This acts as if a user called logcat immediately after the specified logs have been logged, which is particularly useful since it will show the chatty pruning messages at that point. It additionally prints the statistics from logcat -S after the logs. buffers is a comma separated list of the numeric buffer id values from <android/log.h>. For example, 0,1,3 represents the main, radio, and system buffers. It can can also be all. print_point is an positive integer. If it is unspecified, logs are printed after the entire input file is consumed.
  5. nothing BUFFER_TYPE - this does nothing other than read the input file and call Log() for the given buffer type. This is used for profiling CPU usage of strictly the log buffer.