Can an array(union()) schema filter out invalid elements, rather than fail?

[drooJohnson]

I don't know if this is an option within valibot or if we simply need to do this sort of data transformation before we hand the data to valibot to parse. Is there any way to allow an array(union([])) to kick invalid array entries from the output array rather than fail to parse.

For example:

const blockOneSchema = object({
	type: literal("blockOne"),
	id: string(),
	color: picklist(["blue", "red"])
});

const blockTwoSchema = object({
	type: literal("blockTwo"),
	id: string(),
	text: string()
});

const entrySchema = object({
	name: string(),
	blocks: array(union([blockOneSchema, blockTwoSchema]))
});

const exampleData = {
	name: "entry",
	blocks: [{
		type: "blockOne",
		id: "1",
		color: "blue"
	}, {
		type: "blockTwo",
		id: "2",
		text: "stringValue"
	}, {
		type: "blockThree",
		id: "3",
		value: "This isn't in the union, oh no"
	}]
}

const outputData = parse(entrySchema, exampleData);

Basically I want the output to include the first two blocks, since they are valid according to our schema, and instead of failing to parse once it hits the third block — which isn't valid — it would simply remove it from the output array and give us all the valid blocks without a parse failure:

{
	name: "entry",
	blocks: [{
		type: "blockOne",
		id: "1",
		color: "blue"
	}, {
		type: "blockTwo",
		id: "2",
		text: "stringValue"
	}
}

Apologies if this is super wordy, but I'd appreciate any insight about how we can accomplish this.

Just a note, I don't believe I can use variant() instead of union() in our actual code, because the literal we're using to differentiate each type is nested a few levels deep, and it doesn't look like there's any way to target a deeply nested discriminator key.

[fabian-hiller]

Just a note, I don't believe I can use variant() instead of union() in our actual code, because the literal we're using to differentiate each type is nested a few levels deep, and it doesn't look like there's any way to target a deeply nested discriminator key.

Try targeting the root key that contains the nested key. This should work.

Can an array(union()) schema filter out invalid elements, rather than fail?

Currently this functionality is not supported by default. Since Valibot follows a modular architecture, it can be easily extended with custom code. If you want, I can provide the code for a filterArray function that does exactly what you want. If more users are interested in this feature, I will consider adding such a function to the source code of the library.

[drooJohnson]

This was extremely helpful, thank you! The variant does seem to work just fine even if it isn't targeting the exact key for the discriminator, beautiful. If you could provide the code for the filterArray function, and show me where it would go in the schema from my original post, I would greatly appreciate it.

[fabian-hiller]

For filterArray you can basically copy the source code of the array function and modify it a bit but as we are still pre v1 there will be breaking changes from time to time.

import {
  type BaseSchema,
  type ErrorMessage,
  type Input,
  type Output,
  type Pipe,
  defaultArgs,
  pipeResult,
  schemaIssue
} from 'valibot';

/**
 * Filter array schema type.
 */
export interface FilterArraySchema<
  TItem extends BaseSchema,
  TOutput = Output<TItem>[],
> extends BaseSchema<Input<TItem>[], TOutput> {
  /**
   * The schema type.
   */
  type: 'array';
  /**
   * The array item schema.
   */
  item: TItem;
  /**
   * The error message.
   */
  message: ErrorMessage | undefined;
  /**
   * The validation and transformation pipeline.
   */
  pipe: Pipe<Output<TItem>[]> | undefined;
}

/**
 * Creates a filter array schema.
 *
 * @param item The item schema.
 * @param pipe A validation and transformation pipe.
 *
 * @returns A array schema.
 */
export function filterArray<TItem extends BaseSchema>(
  item: TItem,
  pipe?: Pipe<Output<TItem>[]>
): FilterArraySchema<TItem>;

/**
 * Creates a filter array schema.
 *
 * @param item The item schema.
 * @param message The error message.
 * @param pipe A validation and transformation pipe.
 *
 * @returns A array schema.
 */
export function filterArray<TItem extends BaseSchema>(
  item: TItem,
  message?: ErrorMessage,
  pipe?: Pipe<Output<TItem>[]>
): FilterArraySchema<TItem>;

export function filterArray<TItem extends BaseSchema>(
  item: TItem,
  arg2?: ErrorMessage | Pipe<Output<TItem>[]>,
  arg3?: Pipe<Output<TItem>[]>
): FilterArraySchema<TItem> {
  // Get message and pipe argument
  const [message, pipe] = defaultArgs(arg2, arg3);

  // Create and return array schema
  return {
    type: 'array',
    expects: 'Array',
    async: false,
    item,
    message,
    pipe,
    _parse(input, config) {
      // If root type is valid, check nested types
      if (Array.isArray(input)) {
        // Create output array
        const output: any[] = [];

        // Parse schema of each array item
        for (let key = 0; key < input.length; key++) {
          const value = input[key];
          const result = this.item._parse(value, config);

          // Add item to output if no issues
          if (!result.issues) {
            output.push(result.output);
          }
        }

        // Return pipe result
        return pipeResult(this, output as Output<TItem>[], config);
      }

      // Otherwise, return schema issue
      return schemaIssue(this, filterArray, input, config);
    },
  };
}

[drooJohnson]

@fabian-hiller Hey! Is there any chance you could point me in the right direction to update this code to the 1.39 equivalent?

[fabian-hiller]

This should work:

import type {
  _addIssue,
  ArrayIssue,
  BaseIssue,
  BaseSchema,
  Dataset,
  ErrorMessage,
  InferInput,
  InferIssue,
  InferOutput,
} from 'valibot';

/**
 * Filter array schema type.
 */
export interface FilterArraySchema<
  TItem extends BaseSchema<unknown, unknown, BaseIssue<unknown>>,
  TMessage extends ErrorMessage<ArrayIssue> | undefined,
> extends BaseSchema<
    InferInput<TItem>[],
    InferOutput<TItem>[],
    ArrayIssue | InferIssue<TItem>
  > {
  /**
   * The schema type.
   */
  readonly type: 'filter_array';
  /**
   * The schema reference.
   */
  readonly reference: typeof filterArray;
  /**
   * The expected property.
   */
  readonly expects: 'Array';
  /**
   * The array item schema.
   */
  readonly item: TItem;
  /**
   * The error message.
   */
  readonly message: TMessage;
}

/**
 * Creates a filter array schema.
 *
 * @param item The item schema.
 *
 * @returns A filtered array schema.
 */
export function filterArray<
  const TItem extends BaseSchema<unknown, unknown, BaseIssue<unknown>>,
>(item: TItem): FilterArraySchema<TItem, undefined>;

/**
 * Creates a filter array schema.
 *
 * @param item The item schema.
 * @param message The error message.
 *
 * @returns A filtered array schema.
 */
export function filterArray<
  const TItem extends BaseSchema<unknown, unknown, BaseIssue<unknown>>,
  const TMessage extends ErrorMessage<ArrayIssue> | undefined,
>(item: TItem, message: TMessage): FilterArraySchema<TItem, TMessage>;

export function filterArray(
  item: BaseSchema<unknown, unknown, BaseIssue<unknown>>,
  message?: ErrorMessage<ArrayIssue>
): FilterArraySchema<
  BaseSchema<unknown, unknown, BaseIssue<unknown>>,
  ErrorMessage<ArrayIssue> | undefined
> {
  return {
    kind: 'schema',
    type: 'filter_array',
    reference: filterArray,
    expects: 'Array',
    async: false,
    item,
    message,
    _run(dataset, config) {
      // Get input value from dataset
      const input = dataset.value;

      // If root type is valid, check nested types
      if (Array.isArray(input)) {
        // Set typed to `true` and value to empty array
        dataset.typed = true;
        dataset.value = [];

        // Parse schema of each array item
        for (let key = 0; key < input.length; key++) {
          const value = input[key];
          const itemDataset = this.item._run({ typed: false, value }, config);

          // Add item to dataset if valid
          if (!itemDataset.issues) {
            // @ts-expect-error
            dataset.value.push(itemDataset.value);
          }
        }

        // Otherwise, add array issue
      } else {
        _addIssue(this, 'type', dataset, config);
      }

      // Return output dataset
      return dataset as Dataset<unknown[], ArrayIssue | BaseIssue<unknown>>;
    },
  };
}