Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: improved logging #89

Open
wants to merge 4 commits into
base: development
Choose a base branch
from
Open

feat: improved logging #89

wants to merge 4 commits into from

Conversation

adubovik
Copy link
Collaborator

@adubovik adubovik commented Dec 27, 2024 •
edited
Loading

  1. Removed Batches: 100%|███████████████| 1/1 [00:00<00:00, 1.44it/s] tqdm progress bars printed on each request
  2. Removed Chat completion response length {int} info logs printed on each request
  3. Added logger and timings to the individual stages of message processing: topic, lang_id, influx
  4. Used the same log format that is used in adapters
  5. Added LOG_LEVEL env var that resolves Log level management #54

Before:

INFO:     Loading environment from '.env'
2024-12-27 14:20:29,709 [INFO] - Started server process [83620]
2024-12-27 14:20:29,709 [INFO] - Waiting for application startup.
2024-12-27 14:20:30,255 [INFO] - Load pretrained SentenceTransformer: all-mpnet-base-v2
2024-12-27 14:20:31,684 [INFO] - Use pytorch device_name: mps
Batches[encode]: 100%|███████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.74it/s]
2024-12-27 14:20:32,640 [INFO] - Application startup complete.
2024-12-27 14:20:32,640 [INFO] - Uvicorn running on http://127.0.0.1:5006 (Press CTRL+C to quit)
...
2024-12-27 14:20:58,702 [INFO] - Chat completion response length 4
Batches[encode]: 100%|███████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  3.35it/s]
2024-12-27 14:20:59,028 [INFO] - Chat completion response length 4
Batches[encode]: 100%|███████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  4.93it/s]
2024-12-27 14:20:59,244 [INFO] - Chat completion response length 4
Batches[encode]: 100%|███████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  3.33it/s]
2024-12-27 14:20:59,552 [INFO] - Chat completion response length 4
Batches[encode]: 100%|███████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  2.89it/s]
2024-12-27 14:20:59,943 [INFO] - Chat completion response length 4
Batches[encode]: 100%|███████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  6.47it/s]
2024-12-27 14:21:00,116 [INFO] - 127.0.0.1:62705 - "POST /data HTTP/1.1" 200
2024-12-27 14:21:00,117 [INFO] - 127.0.0.1:62708 - "POST /data HTTP/1.1" 200
2024-12-27 14:21:00,117 [INFO] - 127.0.0.1:62699 - "POST /data HTTP/1.1" 200
2024-12-27 14:21:00,117 [INFO] - 127.0.0.1:62701 - "POST /data HTTP/1.1" 200
2024-12-27 14:21:00,130 [INFO] - 127.0.0.1:62704 - "POST /data HTTP/1.1" 200
2024-12-27 14:21:00,130 [INFO] - 127.0.0.1:62703 - "POST /data HTTP/1.1" 200
2024-12-27 14:21:00,131 [INFO] - 127.0.0.1:62700 - "POST /data HTTP/1.1" 200
2024-12-27 14:21:00,131 [INFO] - 127.0.0.1:62709 - "POST /data HTTP/1.1" 200
2024-12-27 14:21:00,133 [INFO] - 127.0.0.1:62702 - "POST /data HTTP/1.1" 200
2024-12-27 14:21:00,134 [INFO] - 127.0.0.1:62710 - "POST /data HTTP/1.1" 200

After:

INFO:     Loading environment from '.env'
INFO:     | 2024-12-27 14:15:27,123 | 40334 | uvicorn.error | Started server process [40334]
INFO:     | 2024-12-27 14:15:27,124 | 40334 | uvicorn.error | Waiting for application startup.
INFO:     | 2024-12-27 14:15:28,204 | 40334 | sentence_transformers.SentenceTransformer | Load pretrained SentenceTransformer: all-mpnet-base-v2
INFO:     | 2024-12-27 14:15:29,233 | 40334 | sentence_transformers.SentenceTransformer | Use pytorch device_name: mps
INFO:     | 2024-12-27 14:15:30,463 | 40334 | uvicorn.error | Application startup complete.
INFO:     | 2024-12-27 14:15:30,480 | 40334 | uvicorn.error | Uvicorn running on http://127.0.0.1:5006 (Press CTRL+C to quit)
...
INFO:     | 2024-12-27 14:18:42,464 | 40334 | app | number of messages: 2
DEBUG:    | 2024-12-27 14:18:42,567 | 40334 | app | [2/2][topic] 0.151s
DEBUG:    | 2024-12-27 14:18:42,568 | 40334 | app | [2/2][langid] 0.004s
DEBUG:    | 2024-12-27 14:18:42,647 | 40334 | app | [1/2][influx] 2.377s
INFO:     | 2024-12-27 14:18:42,879 | 40334 | app | [1/2] success
DEBUG:    | 2024-12-27 14:18:42,934 | 40334 | app | [1/2] 2.721s
...
DEBUG:    | 2024-12-27 14:18:43,242 | 40334 | app | [2/2][influx] 0.558s
INFO:     | 2024-12-27 14:18:43,256 | 40334 | app | [2/2] success
DEBUG:    | 2024-12-27 14:18:43,345 | 40334 | app | [2/2] 0.758s
DEBUG:    | 2024-12-27 14:18:43,345 | 40334 | app | 2.676s
DEBUG:    | 2024-12-27 14:18:43,566 | 40334 | app | response: [{"status": "success"}, {"status": "success"}]
INFO:     | 2024-12-27 14:18:43,678 | 40334 | uvicorn.access | 127.0.0.1:62587 - "POST /data HTTP/1.1" 200

@adubovik adubovik self-assigned this Dec 27, 2024
@adubovik adubovik requested a review from Allob as a code owner December 27, 2024 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Allob Allob Awaiting requested review from Allob Allob is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

@adubovik adubovik

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Log level management
1 participant
@adubovik