Skip to content

Commit

Permalink
!!!FEATURE: Reform i18n mechanism (#3804)
Browse files Browse the repository at this point in the history
Prelude to: #3773
refs: #3119

This PR reforms the UI's i18n mechanism by exposing a new function `translate` directly from the `@neos-project/neos-ui-i18n` package.

The function looks as follows:
```typescript
function translate(
    fullyQualifiedTranslationAddressAsString: string,
    fallback: string | [string, string],
    parameters: Parameters = [],
    quantity: number = 0
): string;
```

And can be used like this:
```typescript
translate(
  'Neos.Neos:Main:workspaces.allChangesInWorkspaceHaveBeenDiscarded',
  'All changes from workspace "{0}" have been discarded.',
  ['user-admin']
);

// output (en): All changes from workspace "user-admin" have been discarded.
```

The new mechanism is completely independent from registries. There's no longer a need for injecting the `I18nRegistry` to use translations, instead `translate` can be used directly.

`translate` will only work after the translations have been loaded during the bootstrapping process of `@neos-project/neos-ui`. If it is used too early, it throws an exception.

The function now also properly supports plural rules (via [`Intl` Web API](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl)). Tha `I18nRegistry` had only supported English plurals, but with broken rules (quantity=0 yielded singular instead of plural).

Other changes included in this PR are:

- Everything in `@neos-project/neos-ui-i18n` is now TypeScript (save `manifest.js`)
- The `<I18n/>`-component was deprecated
- The `I18nRegistry` is now properly documented, but was also deprecated
- The `xliff.json` endpoint is now discovered via `<link>`-tag, instead of initial data
  - The `<link>`-tag is marked with `prefetch` to slightly speed up the fetch request
- The `@neos-project/neos-ui-i18n` now has a comprehensive README.md

This change is breaking, because translations may have relied on the broken plural rules. It is also breaking, because the signature for substitution parameters has changed slightly - although this should be non-breaking at runtime.
  • Loading branch information
grebaldi authored Jan 14, 2025
1 parent f6a552f commit 5272033
Show file tree
Hide file tree
Showing 64 changed files with 2,776 additions and 404 deletions.
5 changes: 4 additions & 1 deletion .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ module.exports = {
'default-case': 'off',
'no-mixed-operators': 'off',
'no-negated-condition': 'off',
'complexity': 'off'
'complexity': 'off',

// This rule would prevent us from implementing meaningful value objects
'no-useless-constructor': 'off'
},
}
Binary file not shown.
Binary file not shown.
14 changes: 14 additions & 0 deletions Classes/Presentation/ApplicationView.php
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@

use Neos\Flow\Annotations as Flow;
use Neos\Flow\Core\Bootstrap;
use Neos\Flow\I18n\Cldr\Reader\PluralsReader;
use Neos\Flow\I18n\Locale;
use Neos\Flow\Mvc\View\AbstractView;
use Neos\Flow\ResourceManagement\ResourceManager;
use Neos\Flow\Security\Context as SecurityContext;
Expand Down Expand Up @@ -49,6 +51,9 @@ final class ApplicationView extends AbstractView
#[Flow\Inject]
protected Bootstrap $bootstrap;

#[Flow\Inject]
protected PluralsReader $pluralsReader;

/**
* This contains the supported options, their default values, descriptions and types.
*
Expand Down Expand Up @@ -113,6 +118,15 @@ private function renderHead(): string
)
);

$locale = new Locale($this->userService->getInterfaceLanguage());
// @TODO: All endpoints should be treated this way and be isolated from
// initial data.
$result .= sprintf(
'<link id="neos-ui-uri:/neos/xliff.json" rel="prefetch" href="%s" data-locale="%s" data-locale-plural-rules="%s">',
$this->variables['initialData']['configuration']['endpoints']['translations'],
(string) $locale,
implode(',', $this->pluralsReader->getPluralForms($locale)),
);
$result .= sprintf(
'<script id="initialData" type="application/json">%s</script>',
json_encode($this->variables['initialData']),
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
"@neos-project/eslint-config-neos": "^2.6.1",
"@typescript-eslint/eslint-plugin": "^5.44.0",
"@typescript-eslint/parser": "^5.44.0",
"cross-fetch": "^4.0.0",
"editorconfig-checker": "^4.0.2",
"esbuild": "~0.17.0",
"eslint": "^8.27.0",
Expand Down
3 changes: 1 addition & 2 deletions packages/jest-preset-neos-ui/src/setupBrowserEnv.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
import 'regenerator-runtime/runtime';
import browserEnv from 'browser-env';
import 'cross-fetch/polyfill';

browserEnv();

window.fetch = () => Promise.resolve(null);
3 changes: 3 additions & 0 deletions packages/neos-ts-interfaces/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@
"description": "Neos domain-related TypeScript interfaces",
"private": true,
"main": "src/index.ts",
"dependencies": {
"@neos-project/neos-ui-i18n": "workspace:*"
},
"devDependencies": {
"@neos-project/jest-preset-neos-ui": "workspace:*",
"typescript": "^4.6.4"
Expand Down
7 changes: 4 additions & 3 deletions packages/neos-ts-interfaces/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import type {I18nRegistry} from '@neos-project/neos-ui-i18n';

export type NodeContextPath = string;
export type FusionPath = string;
export type NodeTypeName = string;
Expand Down Expand Up @@ -268,10 +270,9 @@ export interface ValidatorRegistry {
get: (validatorName: string) => Validator | null;
set: (validatorName: string, validator: Validator) => void;
}
export interface I18nRegistry {
translate: (id?: string, fallback?: string, params?: {}, packageKey?: string, sourceName?: string) => string;
}
export interface GlobalRegistry {
get: <K extends string>(key: K) => K extends 'i18n' ? I18nRegistry :
K extends 'validators' ? ValidatorRegistry : null;
}

export type {I18nRegistry} from '@neos-project/neos-ui-i18n';
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import {processSelectBoxOptions} from './selectBoxHelpers';
import {I18nRegistry} from '@neos-project/neos-ts-interfaces';

const fakeI18NRegistry: I18nRegistry = {
const fakeI18NRegistry = {
translate: (id) => id ?? ''
};
} as I18nRegistry;

describe('processSelectBoxOptions', () => {
it('transforms an associative array with labels to list of objects', () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import React from 'react';
import Logo from '@neos-project/react-ui-components/src/Logo';
import Button from '@neos-project/react-ui-components/src/Button';
import Icon from '@neos-project/react-ui-components/src/Icon';
import {I18nRegistry} from '@neos-project/neos-ts-interfaces';
import type {I18nRegistry} from '@neos-project/neos-ui-i18n';

import styles from './style.module.css';

Expand Down
244 changes: 244 additions & 0 deletions packages/neos-ui-i18n/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,244 @@
# @neos-project/neos-ui-i18n

> I18n utilities for Neos CMS UI.
This package connects Flow's Internationalization (I18n) framework with the Neos UI.

In Flow, translations are organized in [XLIFF](http://en.wikipedia.org/wiki/XLIFF) files that are stored in the `Resources/Private/Translations/`-folder of each Flow package.

The Neos UI does not load all translation files at once, but only those that have been made discoverable explicitly via settings:
```yaml
Neos:
Neos:
userInterface:
translation:
autoInclude:
'Neos.Neos.Ui':
- Error
- Main
// ...
'Vendor.Package':
- Main
// ...
```

At the beginning of the UI bootstrapping process, translations are loaded from an enpoint (see: [`\Neos\Neos\Controller\Backend\BackendController->xliffAsJsonAction()`](https://neos.github.io/neos/9.0/Neos/Neos/Controller/Backend/BackendController.html#method_xliffAsJsonAction)) and are available afterwards via the `translate` function exposed by this package.

## API

### `translate`

```typescript
function translate(
fullyQualifiedTranslationAddressAsString: string,
fallback: string | [string, string],
parameters: Parameters = [],
quantity: number = 0
): string;
```

`translate` will use the given translation address to look up a translation from the ones that are currently available (see: [`initializeI18n`](#initializeI18n)).

To understand how the translation address maps onto the translations stored in XLIFF files, let's take a look at the structure of the address:
```
"Neos.Neos.Ui:Main:errorBoundary.title"
└────┬─────┘ └─┬┘ └───────────┬─────┘
Package Key Source Name trans-unit ID
```

Each translation address consists of three Parts, one identifying the package (Package Key), one identifying the XLIFF file (Source Name), and one identifying the translation itself within the XLIFF file (trans-unit ID).

Together with the currently set `Locale`, Package Key and Source Name identify the exact XLIFF file for translation thusly:
```
resource://{Package Key}/Private/Translations/{Locale}/{Source Name}.xlf
```

So, the address `Neos.Neos.Ui:Main:errorBoundary.title` would lead us to:
```
resource://Neos.Neos.Ui/Private/Translations/de/Main.xlf
```

Within the XLIFF-file, the trans-unit ID identifies the exact translation to be used:
```xml
<?xml version="1.0" encoding="UTF-8"?>
<xliff version="1.2" xmlns="urn:oasis:names:tc:xliff:document:1.2">
<file original="" product-name="Neos.Neos.Ui" source-language="en" target-language="de" datatype="plaintext">
<body>
<!-- ... -->
<!-- ↓ This is the one -->
<trans-unit id="errorBoundary.title" xml:space="preserve">
<source>Sorry, but the Neos UI could not recover from this error.</source>
<target>Es tut uns leid, aber die Neos Benutzeroberfläche konnte von diesem Fehler nicht wiederhergestellt werden.</target>
</trans-unit>
<!-- ... -->
</body>
</file>
</xliff>
```

If no translation can be found, `translate` will return the given `fallback` string.

Translations (and fallbacks) may contain placeholders, like:
```
All changes from workspace "{0}" have been discarded.
```

Placeholders may be numerically indexed (like the one above), or indexed by name, like:
```
Copy {source} to {target}
```

For numerically indexed placeholders, you can pass an array of strings to the `parameters` argument of `translate`. For named parameters, you can pass an object with string values and keys identifying the parameters.

Translations may also have plural forms. `translate` uses the [`Intl` Web API](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl) to pick the currect plural form for the current `Locale` based on the given `quantity`.

Fallbacks can also provide plural forms, but will always treated as if we're in locale `en-US`, so you can only provide two different plural forms.

#### Arguments

| Name | Description |
|-|-|
| `fullyQualifiedTranslationAddressAsString` | The translation address for the translation to use, e.g.: `"Neos.Neos.Ui:Main:errorBoundary.title"` |
| `fallback` | The string to return, if no translation can be found under the given address. If a tuple of two strings is passed here, these will be treated as singular and plural forms of the translation. |
| `parameters` | Values to replace placeholders in the translation with. This can be passed as an array of strings (to replace numerically indexed placeholders) or as a `Record<string, string>` (to replace named placeholders) |
| `quantity` | The quantity is used to determine which plural form (if any) to use for the translation |

#### Examples

##### Translation without placeholders or plural forms

```typescript
translate('Neos.Neos.Ui:Main:insert', 'insert');
// output (en): "insert"
```

##### Translation with a numerically indexed placeholder

```typescript
translate(
'Neos.Neos:Main:workspaces.allChangesInWorkspaceHaveBeenDiscarded',
'All changes from workspace "{0}" have been discarded.',
['user-admin']
);

// output (en): All changes from workspace "user-admin" have been discarded.
```

##### Translation with a named placeholder

```typescript
translate(
'Neos.Neos.Ui:Main:deleteXNodes',
'Delete {amount} nodes',
{amount: 12}
);

// output (en): "Delete 12 nodes"
```

##### Translations with placeholders and plural forms

```typescript
translate(
'Neos.Neos.Ui:Main:changesPublished',
['Published {0} change to "{1}".', 'Published {0} changes to "{1}".']
[1, "live"],
1
);
// output (en): "Published 1 change to "live"."

translate(
'Neos.Neos.Ui:Main:changesPublished',
['Published {0} change to "{1}".', 'Published {0} changes to "{1}".']
[20],
20
);
// output (en): "Published 20 changes to "live"."
```

### `initializeI18n`

```typescript
async function initializeI18n(): Promise<void>;
```

> [!NOTE]
> Usually you won't have to call this function yourself. The Neos UI will
> set up I18n automatically.
This function loads the translations from the translations endpoint and makes them available globally. It must be run exactly once before any call to `translate`.

The exact URL of the translations endpoint is discoverd via the DOM. The document needs to have a link tag with the id `neos-ui-uri:/neos/xliff.json`, with the following attributes:
```html
<link
id="neos-ui-uri:/neos/xliff.json"
href="https://mysite.example/neos/xliff.json?locale=de-DE"
data-locale="de-DE"
data-locale-plural-rules="one,other"
>
```

The `ApplicationView` PHP class takes care of rendering this tag.

### `setupI18n`

```typescript
function setupI18n(
localeIdentifier: string,
pluralRulesAsString: string,
translations: TranslationsDTO
): void;
```

This function can be used in unit tests to set up I18n.

#### Arguments

| Name | Description |
|-|-|
| `localeIdentifier` | A valid [Unicode Language Identifier](https://www.unicode.org/reports/tr35/#unicode-language-identifier), e.g.: `de-DE`, `en-US`, `ar-EG`, ... |
| `pluralRulesAsString` | A comma-separated list of [Language Plural Rules](http://www.unicode.org/reports/tr35/#Language_Plural_Rules) matching the locale specified by `localeIdentifier`. Here, the output of [`\Neos\Flow\I18n\Cldr\Reader\PluralsReader->getPluralForms()`](https://neos.github.io/flow/9.0/Neos/Flow/I18n/Cldr/Reader/PluralsReader.html#method_getPluralForms) is expected, e.g.: `one,other` for `de-DE`, or `zero,one,two,few,many` for `ar-EG` |
| `translations` | The XLIFF translations in their JSON-serialized form |

##### `TranslationsDTO`

```typescript
type TranslationsDTO = {
[serializedPackageKey: string]: {
[serializedSourceName: string]: {
[serializedTransUnitId: string]: string | string[]
}
}
}
```
The `TranslationDTO` is the payload of the response from the translations endpoint (see: [`\Neos\Neos\Controller\Backend\BackendController->xliffAsJsonAction()`](https://neos.github.io/neos/9.0/Neos/Neos/Controller/Backend/BackendController.html#method_xliffAsJsonAction)).
###### Example:
```jsonc
{
"Neos_Neos_Ui": { // <- Package Key with "_" instead of "."
"Main": { // <- Source name with "_" instead of "."

// Example without plural forms
"errorBoundary_title": // <- trans-unit ID with "_" instead of "."
"Sorry, but the Neos UI could not recover from this error.",

// Example with plural forms
"changesDiscarded": [ // <- trans-unit ID with "_" instead of "."
"Discarded {0} change.",
"Discarded {0} changes."
]
}
}
}
```
### `teardownI18n`
```typescript
function teardownI18n(): void;
```

This function must be used in unit tests to clean up when `setupI18n` has been used.
4 changes: 1 addition & 3 deletions packages/neos-ui-i18n/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,13 @@
"version": "",
"description": "I18n utilities and components for Neos CMS UI.",
"private": true,
"main": "./src/index.tsx",
"main": "./src/index.ts",
"devDependencies": {
"@neos-project/jest-preset-neos-ui": "workspace:*",
"enzyme": "^3.8.0",
"typescript": "^4.6.4"
},
"dependencies": {
"@neos-project/neos-ts-interfaces": "workspace:*",
"@neos-project/neos-ui-decorators": "workspace:*",
"@neos-project/neos-ui-extensibility": "workspace:*",
"@neos-project/utils-logger": "workspace:*"
},
Expand Down
Loading

0 comments on commit 5272033

Please sign in to comment.