WIP: Add initial metrics #573
Conversation
…rently this commit also edits the startHealthServer to also start collecting of default prometheus metrics. channels are used to collect potential errors and allow for graceful shutdown of servers
…shutdown once jobctrl is shut down
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Thanks @eranra for the PR. If I understand the PR correctly these changes would add a duplicate option to externalizing GPU metrics. It would be good to discuss a plugin interface for externalizing metrics such that the existing, and alternative metrics externalization implementations can be configured for the MCAD controller.
| @@ -80,7 +81,9 @@ func (s *ServerOption) AddFlags(fs *flag.FlagSet) { | |||
| fs.BoolVar(&s.QuotaEnabled, "quotaEnabled", s.QuotaEnabled, "Enable quota policy evaluation. Default is false.") | |||
| fs.StringVar(&s.QuotaRestURL, "quotaURL", s.QuotaRestURL, "URL for ReST quota management. Default is none.") | |||
| fs.IntVar(&s.SecurePort, "secure-port", 6443, "The port on which to serve secured, authenticated access for metrics.") | |||
| fs.StringVar(&s.HealthProbeListenAddr, "healthProbeListenAddr", ":8081", "Listen address for health probes. Defaults to ':8081'") | |||
| fs.IntVar(&s.HealthProbeListenPort, "healthProbeListenPort", 8081, "Listen port for health probes. Defaults to ':8081'") | |||
There was a problem hiding this comment.
@eranra Helm chart may be producing a string here. Could you check and validate/fix helm chart. See deployment/mcad-controller
|
@eranra Can you open up an ADR for this PR, please? |
|
PR needs rebase. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Issue link
What changes have been made
This PR adds a basic set of initial metrics to be exposed by MCAD. The metrics are in Prometheus format and will provide the same Observability information we have today using the
fake adaptormechanism.This PR is based on PR #486
Verification steps
Checks