Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev json logs #22994

Open
wants to merge 13 commits into
base: 5.x-dev
Choose a base branch
from
Open

Dev json logs #22994

wants to merge 13 commits into from

Conversation

krezreb
Copy link

@krezreb krezreb commented Jan 28, 2025

Description:

Modifies the LineMessageFormatter to support outputting of log messages as json (one json object per line). This has a number of benefits for observability:

  • No more multiline logs
  • Logs can now be parsed out of the box without having to write specific parsers
  • End-Users' ability to poison logs is limited as all logs are properly escaped

Also adds support for customers to add custom json processing to each line (useful for example for adding APM trace and span ids in for Datadog Unified service tagging users)

Review

Depends on plugin PR matomo-org/plugin-LogViewer#99

To test json_logs

  1. Checkout the dev-json-logs branch on Matomo core
  2. Modify config/config.ini.php set enable_json = true (see example below)
[log]
log_writers[] = file
log_level = DEBUG
enable_json = true
logger_file_path = tmp/logs/matomo.log
  1. Tail the the log file and you should see the logs as json
  2. Check the logviewer plugin index.php?module=LogViewer logs should appear correctly in the interface

To test exclude_pattens

  1. Modify config/config.ini.php set one or more exclude_patterns, separated by pipes(see example below)
[log]
log_writers[] = file
log_level = DEBUG
exclude_patterns =  "Piwik\Plugin\Manager|Matomo is not set up"
enable_json = true
logger_file_path = tmp/logs/matomo.log
  1. Tail the the log file and you should see the logs as json, EXCEPT FOR logs matching the exclude patterns

To test custom_function_file

  1. Modify config/config.ini.php specify custom_function_file (see example below)
[log]
log_writers[] = file
log_level = DEBUG
custom_function_file = /var/www/html/config/custom.php
enable_json = true
logger_file_path = tmp/logs/matomo.log
  1. Example custom.php
<?php

function userHandler(array $array) : ?array
{
        $array["foo"] = "bar";
        return $array;
};
# return a string matching the name of the above function
return 'userHandler';
  1. Tail the the log file and you should see the logs as json with the above custom logic run, in the above case a key "foo" will be set to the value "bar"

@patrickli
Copy link

@krezreb krezreb requested review from michalkleiner and a team February 11, 2025 09:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants