Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modified the signature of ITelemetryLogger #813

Merged
merged 42 commits into from
Jan 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
cb8fee0
First cut of telemetry
BidishaMS Jan 4, 2024
1d65e5d
First cut of telemetry
BidishaMS Jan 4, 2024
39c424b
changes in 1DSCollector
BidishaMS Jan 4, 2024
5cf3f79
Merge branch 'main' into users/biddas/clientLoggerWrapper
BidishaMS Jan 4, 2024
5c99e6a
Merge branch 'main' into users/biddas/clientLoggerWrapper
BidishaMS Jan 4, 2024
faecb48
renamed from collector to logger
BidishaMS Jan 4, 2024
7c65e72
Merge branch 'users/biddas/clientLoggerWrapper' of https://github.com…
BidishaMS Jan 4, 2024
e9058b4
merged to remote
BidishaMS Jan 4, 2024
cf47caf
Merge branch 'main' into users/biddas/clientLoggerWrapper
BidishaMS Jan 5, 2024
10975c5
Added wrapper
BidishaMS Jan 8, 2024
6a3f4dc
consumed in client
BidishaMS Jan 9, 2024
9392543
added types
BidishaMS Jan 10, 2024
fc9f264
merged master
BidishaMS Jan 11, 2024
1717d5d
changes
BidishaMS Jan 11, 2024
32ace97
Created common interfaces and wrapper
BidishaMS Jan 11, 2024
59da8c3
reverted back changes
BidishaMS Jan 11, 2024
4628533
removed blank line
BidishaMS Jan 11, 2024
3f745bf
removed blank line
BidishaMS Jan 11, 2024
2fec1e4
removed blank line
BidishaMS Jan 11, 2024
61c5afb
Merge branch 'main' into users/biddas/clientLoggerWrapper1
BidishaMS Jan 11, 2024
ee6954c
Merge branch 'main' into users/biddas/clientLoggerWrapper1
BidishaMS Jan 11, 2024
6775fd6
populate data
BidishaMS Jan 11, 2024
438eb9d
Removed platformType
BidishaMS Jan 11, 2024
bd86c55
Merge branch 'main' of https://github.com/microsoft/powerplatform-vsc…
BidishaMS Jan 11, 2024
a22e26e
Merge branch 'users/biddas/clientLoggerWrapper1' of https://github.co…
BidishaMS Jan 11, 2024
f7d35f8
Merge branch 'users/biddas/clientLoggerWrapper1' into users/biddas/Po…
BidishaMS Jan 11, 2024
f470dc2
Removed platformType
BidishaMS Jan 11, 2024
fab0f38
changes
BidishaMS Jan 15, 2024
3e46ef9
changes
BidishaMS Jan 17, 2024
bfc06ef
changes
BidishaMS Jan 17, 2024
2fec3e7
changes
BidishaMS Jan 17, 2024
544f04f
revert this changes
BidishaMS Jan 17, 2024
751f1a3
revert this changes
BidishaMS Jan 17, 2024
f854efe
Modifying OneDSLogger API
BidishaMS Jan 17, 2024
1e8195b
Modifying OneDSLogger API
BidishaMS Jan 17, 2024
4af6d02
removed cloudRoleInstance
BidishaMS Jan 18, 2024
3836c62
fixed table name
BidishaMS Jan 18, 2024
2e2d714
changes
BidishaMS Jan 19, 2024
a73b346
changes
BidishaMS Jan 19, 2024
01e5024
changes
BidishaMS Jan 19, 2024
81762d0
changes
BidishaMS Jan 19, 2024
535affe
Merge branch 'main' into users/biddas/ModifyAPIs
BidishaMS Jan 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions src/common/OneDSLoggerTelemetry/EventContants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ export enum CustomType {

}

export enum EventTableName {
CUSTOM_EVENT = 'CustomEvent',
export enum OneDSCollectorEventName {
VSCODE_EVENT = 'VscodeEvent',
REDACTED = "-REDACTED-"
}
8 changes: 4 additions & 4 deletions src/common/OneDSLoggerTelemetry/ITelemetryLogger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,21 +8,21 @@ export interface ITelemetryLogger {
* Actual implementation that send telemetry event
* @param eventName - Telemetry event to send over
*/
traceInfo(eventName:string, customDimension?:Record<string, string>, customMeasurement?: Record<string, number>, message?:string) : void
traceInfo(eventName:string, eventInfo?:object, measurement?: object) : void

/**
* Send warning telemetry event
* @param eventName - Event to send
*/
traceWarning(eventName:string, customDimension?:Record<string, string>, customMeasurement?: Record<string, number>, message?:string) : void
traceWarning(eventName:string, eventInfo?:object, measurement?: object) : void
/**
* Send error telemetry event
* @param eventName - Event to send
*/
traceError(eventName: string, error?:Error, customDimension?:Record<string, string>, customMeasurement?: Record<string, number>, exceptionMessage?:string, exceptionSource?:string, exceptionDetails?:string) : void
traceError(eventName: string, errorMessage: string, exception: Error, eventInfo?:object , measurement?: object) : void
/**
* Send featureName and eventName telemetry event
* @param eventName - Event to send
*/
featureUsage(featureName: string, eventName: string, customDimensions?: Record<string, string>) : void
featureUsage(featureName: string, eventName: string, eventInfo?:object) : void
}
156 changes: 117 additions & 39 deletions src/common/OneDSLoggerTelemetry/oneDSLogger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,16 @@
* Licensed under the MIT License. See License.txt in the project root for license information.
*/

/* eslint-disable @typescript-eslint/no-non-null-assertion*/

import { AppInsightsCore, type IExtendedConfiguration } from "@microsoft/1ds-core-js";
import { PostChannel, type IChannelConfiguration, type IXHROverride } from "@microsoft/1ds-post-js";
import { ITelemetryLogger } from "./ITelemetryLogger";
import { EventType } from "./telemetryConstants";
import { EventType, Severity } from "./telemetryConstants";
import * as vscode from "vscode";
import {getExtensionType, getExtensionVersion} from "../../common/Utils";
import { EXTENSION_ID } from "../../client/constants";
import {EventTableName} from "./EventContants";
import {OneDSCollectorEventName} from "./EventContants";

interface IInstrumentationSettings {
endpointURL: string;
Expand All @@ -22,7 +24,11 @@ export class OneDSLogger implements ITelemetryLogger{
private readonly appInsightsCore :AppInsightsCore;
private readonly postChannel: PostChannel;


private readonly regexPatternsToRedact = [
/key["\\ ']*[:=]+["\\ ']*([a-zA-Z0-9]*)/igm,
/token["\\ ']*[:=]+["\\ ']*([a-zA-Z0-9]*)/igm,
/session["\\ ']*[:=]+["\\ ']*([a-zA-Z0-9]*)/igm
]

private readonly fetchHttpXHROverride: IXHROverride = {
sendPOST: (payload, oncomplete) => {
Expand Down Expand Up @@ -101,9 +107,7 @@ export class OneDSLogger implements ITelemetryLogger{
this.appInsightsCore.initialize(coreConfig, []);
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
this.appInsightsCore.addTelemetryInitializer((envelope: any) => {
OneDSLogger.populateCommonAttributes(envelope);
});
this.appInsightsCore.addTelemetryInitializer(this.populateCommonAttributes());
}

private static getInstrumentationSettings(geo?:string): IInstrumentationSettings {
Expand Down Expand Up @@ -146,49 +150,50 @@ export class OneDSLogger implements ITelemetryLogger{
}

/// Trace info log
public traceInfo(eventName:string, customDimension?:Record<string, string>, customMeasurement?: Record<string, number>, message?:string) {
public traceInfo(eventName:string, eventInfo?:object, measurement?: object) {
const event = {
name: EventTableName.CUSTOM_EVENT,
name: OneDSCollectorEventName.VSCODE_EVENT,
data: {
eventName: eventName,
eventType: EventType.INFO,
message: message,
customDimension: customDimension,
customMeasurement: customMeasurement
eventType: EventType.TRACE,
severity: Severity.INFO,
eventInfo: JSON.stringify(eventInfo!),
measurement: JSON.stringify(measurement!)
}
};

this.appInsightsCore.track(event);
}

/// Trace warning log
public traceWarning(eventName:string, customDimension?:Record<string, string>, customMeasurement?: Record<string, number>, message?:string) {
public traceWarning(eventName:string, eventInfo?: object, measurement?: object) {
const event = {
name: EventTableName.CUSTOM_EVENT,
name: OneDSCollectorEventName.VSCODE_EVENT,
data: {
eventName: eventName,
eventType: EventType.WARNING,
message: message,
customDimension: customDimension,
customMeasurement: customMeasurement
eventType: EventType.TRACE,
severity: Severity.WARN,
eventInfo: JSON.stringify(eventInfo!),
measurement: JSON.stringify(measurement!)
}
};

this.appInsightsCore.track(event);
}

// Trace error log
public traceError(eventName: string, error?:Error, customDimension?:Record<string, string>, customMeasurement?: Record<string, number>, exceptionMessage?:string, exceptionSource?:string, exceptionDetails?:string) {
public traceError(eventName: string, errorMessage: string, exception: Error, eventInfo?:object, measurement?: object) {
const event = {
name: EventTableName.CUSTOM_EVENT,
name: OneDSCollectorEventName.VSCODE_EVENT,
data: {
eventName: eventName,
eventType: EventType.ERROR,
exceptionMessage: error?.message,
exceptionDetails: exceptionDetails,
exceptionSource: exceptionSource,
customDimension: customDimension,
customMeasurement: customMeasurement
eventType: EventType.TRACE,
severity: Severity.ERROR,
message: errorMessage!,
errorName: exception!,
errorStack: JSON.stringify(exception!),
eventInfo: JSON.stringify(eventInfo!),
measurement: JSON.stringify(measurement!)
}
};
this.appInsightsCore.track(event);
Expand All @@ -201,10 +206,11 @@ export class OneDSLogger implements ITelemetryLogger{
) {

const event = {
name: EventTableName.CUSTOM_EVENT,
name: OneDSCollectorEventName.VSCODE_EVENT,
data: {
eventName: 'Portal_Metrics_Event',
eventType: EventType.INFO,
eventType: EventType.TRACE,
severity: Severity.INFO,
eventInfo: JSON.stringify({
featureName: featureName,
customDimensions: customDimensions,
Expand All @@ -216,18 +222,90 @@ export class OneDSLogger implements ITelemetryLogger{
}

/// Populate attributes that are common to all events
private populateCommonAttributes() {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
return (envelope:any) => {
try {
envelope.data = envelope.data || {}; // create data nested object if doesn't exist already'

envelope.data.clientSessionId = vscode.env.sessionId;
envelope.data.vscodeSurface = getExtensionType();
envelope.data.vscodeMachineId = vscode.env.machineId;
envelope.data.vscodeExtensionName = EXTENSION_ID;
envelope.data.vscodeExtensionVersion = getExtensionVersion();
envelope.data.vscodeVersion = vscode.version;
envelope.data.domain = vscode.env.appHost;
// envelope.data.timestamp = envelope.ext.user.locale;
// envelope.data.userLocale = envelope.ext.user.locale;
// envelope.data.userTimeZone = envelope.ext.loc.tz;
// envelope.data.appLanguage = envelope.ext.app.locale;

// At the end of event enrichment, redact the sensitive data for all the applicable fields
// envelope = this.redactSensitiveDataFromEvent(envelope);
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
catch (exception:any)
{
// Such exceptions are likely if we are trying to process event attributes which don't exist
// In such cases, only add common attributes and current exception details, and avoid processing the event attributes further
// However, do log the baseData of the event along with its name in the exception event that gets sent out in this scenario
console.warn("Caught exception processing the telemetry event: " + envelope.name);
console.warn(exception.message);

this.traceExceptionInEventProcessing();
return false;
}
}
}

//// Redact Sensitive data for the fields susceptible to contain codes/tokens/keys/secrets etc.
//// This is done post event enrichment is complete to not impact the dependencies (if any) on actual values like Uri etc.
// eslint-disable-next-line @typescript-eslint/no-explicit-any
private static populateCommonAttributes(envelope: any) {
envelope.data = envelope.data || {}; // create data nested object if doesn't exist already'

envelope.data.SessionId = vscode.env.sessionId;
envelope.data.surface = getExtensionType();
envelope.data.extensionVersion = getExtensionVersion();
envelope.data.extensionName = EXTENSION_ID;
envelope.data.dataDomain = vscode.env.appHost;
envelope.data.cloudRoleName = vscode.env.appName;
envelope.data.vscodeVersion = vscode.version;
envelope.data.vscodeMachineId = vscode.env.machineId
private redactSensitiveDataFromEvent(envelope:any) {
//Redact sensitive information from suseptible fields
envelope.data.errorStack = this.getAllSensitiveRedactedFromField(envelope.data.errorStack);
return envelope;
}

//// Get redacted value after all sensitive information is redacted
getAllSensitiveRedactedFromField(value:string) {
try {
// Ensure the value is of type string
if (value && typeof value === 'string') {
this.regexPatternsToRedact.forEach((pattern: RegExp) => {
value = this.getRedactedValueViaRegexMatch(value, pattern);
});
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
} catch (exception:any) {
console.warn("Caught exception while processing telemetry data for redaction (if any): " + value);
console.warn(exception.message);
}
return value;
}

//// Get redacted value
getRedactedValueViaRegexMatch(value:string, regexPattern:RegExp) {
let matches;

while ((matches = regexPattern.exec(value)) !== null) {
// This is necessary to avoid infinite loops with zero-width matches
if (matches.index === regexPattern.lastIndex) {
regexPattern.lastIndex++;
}

matches.forEach((match, groupIndex) => {
if (groupIndex == 0) { // Redact the entire matched string
value = value.replace(match, OneDSCollectorEventName.REDACTED); //Replace with string REDACTED
}
});
}
return value;
}

/// Trace exceptions in processing event attributes
private traceExceptionInEventProcessing() {
// TODO : Add exception handling
}

public flushAndTeardown(): void {
Expand Down
15 changes: 8 additions & 7 deletions src/common/OneDSLoggerTelemetry/oneDSLoggerWrapper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,29 +30,30 @@ export class oneDSLoggerWrapper {
}

/// Trace info log
public traceInfo(eventName:string, customDimension?:Record<string, string>, customMeasurement?: Record<string, number>, message?:string) {
public traceInfo(eventName:string, eventInfo?: object, measurement?: object) {
try{
oneDSLoggerWrapper.oneDSLoggerIntance.traceInfo(eventName, customDimension, customMeasurement, message);
oneDSLoggerWrapper.oneDSLoggerIntance.traceInfo(eventName, eventInfo, measurement);
}catch (exception) {
console.warn(exception);
}
}

/// Trace warning log
public traceWarning(eventName:string, customDimension?:Record<string, string>, customMeasurement?: Record<string, number>, message?:string) {
public traceWarning(eventName:string, eventInfo?: object, measurement?: object) {
try{
oneDSLoggerWrapper.oneDSLoggerIntance.traceWarning(eventName, customDimension, customMeasurement, message);
oneDSLoggerWrapper.oneDSLoggerIntance.traceWarning(eventName, eventInfo, measurement);
}catch (exception) {
console.warn(exception);
}
}

/// Trace exception log
public traceError(eventName:string, error?:Error, customDimension?:Record<string, string>, customMeasurement?: Record<string, number>, message?:string) {
public traceError(eventName: string, errorMessage: string, exception: Error, eventInfo?:object, measurement?: object) {
try{
oneDSLoggerWrapper.oneDSLoggerIntance.traceError(eventName, error, customDimension, customMeasurement, message);
oneDSLoggerWrapper.oneDSLoggerIntance.traceError(eventName, errorMessage, exception, eventInfo, measurement);
}catch (exception) {
console.warn(exception);
console.warn("Caught exception processing the telemetry event: " + exception);
console.warn(exception);
}
}

Expand Down
11 changes: 10 additions & 1 deletion src/common/OneDSLoggerTelemetry/telemetryConstants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,16 @@
*/

export enum EventType {
CLICK = 'Click',
CUSTOM = 'Custom',
GENERAL = 'General',
SCENARIO = 'Scenario',
TRACE = 'Trace'
}

export enum Severity {
DEBUG = 'Debug',
INFO = 'Info',
WARNING = 'Warning',
WARN = 'Warning',
ERROR = 'Error'
}
Loading