Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the event processing chain configurable #27

Open
dxsup opened this issue Jan 28, 2022 · 0 comments
Open

Make the event processing chain configurable #27

dxsup opened this issue Jan 28, 2022 · 0 comments
Assignees
Labels
area/collector Issues or PRs related to agent metric collector enhancement New feature or request

Comments

@dxsup
Copy link
Member

dxsup commented Jan 28, 2022

Currently, we build the processing chain by hard coding, which makes changes inconvenient. Every time we add a new component to the chain, we have to modify the source codes and make the building procedure more verbose and difficult to read. What looks like this:

// buildPipeline builds a event processing pipeline based on hard-code.
func (a *Application) buildPipeline() error {
	// TODO: Build pipeline via configuration to implement dependency injection
	// Initialize exporters
	otelExporterFactory := a.componentsFactory.Exporters[otelexporter.Otel]
	otelExporter := otelExporterFactory.NewFunc(otelExporterFactory.Config, a.telemetry.Telemetry)
	// Initialize all processors
	// 1. Kindling Metric Format Processor
	formatProcessorFactory := a.componentsFactory.Processors[kindlingformatprocessor.ProcessorName]
	formatProcessor := formatProcessorFactory.NewFunc(formatProcessorFactory.Config, a.telemetry.Telemetry, otelExporter)
	// 2. Kubernetes metadata processor
	k8sProcessorFactory := a.componentsFactory.Processors[k8sprocessor.K8sMetadata]
	k8sMetadataProcessor := k8sProcessorFactory.NewFunc(k8sProcessorFactory.Config, a.telemetry.Telemetry, formatProcessor)
        // other initialization
        ...
	a.analyzerManager = analyzerManager
	udsReceiverFactory := a.componentsFactory.Receivers[udsreceiver.Uds]
	udsReceiver := udsReceiverFactory.NewFunc(udsReceiverFactory.Config, a.telemetry.Telemetry, analyzerManager)
	a.receiver = udsReceiver
	return nil
}

Although our processing chain is inspired by the pipeline of Opentelemetry-Collector, its building procedure can't be applied to ours directly. Because the pipeline is one direction for every processor while the processing chain is more like a directed acyclic graph in which every node has multiple fan-out directions.

I will work on this issue.

@dxsup dxsup added enhancement New feature or request area/collector Issues or PRs related to agent metric collector labels Jan 28, 2022
@dxsup dxsup self-assigned this Jan 28, 2022
@dxsup dxsup added this to the V1.0 milestone Jan 28, 2022
@dxsup dxsup removed this from the V1.0 milestone Apr 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/collector Issues or PRs related to agent metric collector enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant