v0.12.1

v0.12.1 (2024-02-15)

Documentation

• docs: Added dataset disclaimer (#145) (6b3e71b)

Fix

• fix: Added get_documents to task interface (c4fb354)

Unknown

• Merge branch 'main' of https://github.com/KennethEnevoldsen/Scandinavian-Embedding-Benchmark (24644a7)

• Fix: Added performance metrics for translate and embed (91a2b8a)

v0.12.0

v0.12.0 (2024-02-12)

Unknown

• Merge pull request #137 from KennethEnevoldsen/update-sonar-models

Update and rerun sonar models (28ff5e5)

v0.11.4

v0.11.4 (2024-02-12)

Fix

• fix: Added model type and releae date to model meta

This is to allow the tracking of improvement on SEB over time (49c9f1a)

Unknown

• Merge pull request #140 from KennethEnevoldsen/adding-architecture

fix: Added model type and releae date to model meta (245e881)

• Merge pull request #131 from KennethEnevoldsen/mistral-instructions

Added instructions for all tasks in Mistral E5 (006c253)

• Added instructions for all tasks in Mistral E5 (07c4c95)

v0.11.3

v0.11.3 (2024-02-06)

Documentation

• docs: Updated table w. dataset descriptions (f955379)

• docs: Added across column for coverage

added swapped formalization and task columns (31d9d76)

Fix

• fix: Updated dataset description metadata and script (0e63eb4)

• fix: Update calc. descriptive stats (bdbd552)

Unknown

• Merge pull request #130 from KennethEnevoldsen/docs-update

Update task/dataset descriptions (144f025)

v0.11.2

v0.11.2 (2024-02-05)

Fix

• fix: Updated table generation of the benchmark (c87aed4)

Style

• style: ran linting (489e124)

Unknown

• Merge pull request #127 from KennethEnevoldsen/run-models

Ran most of the models (db1868e)

• remo ve test srcipt (dcbe547)

• Added translate and embed scores (26805ad)

• Merge branch 'run-models' of https://github.com/KennethEnevoldsen/Scandinavian-Embedding-Benchmark into run-models (5570776)

• Added fasttext and translate scores (d725b44)

v0.11.1

v0.11.1 (2024-02-04)

Documentation

• docs: restructured dataset overview (30440f0)

Fix

• fix: Updated tables (d30284c)

Style

• style: Applied linter (2b0342d)

Unknown

• Merge pull request #121 from KennethEnevoldsen/update-tables

Updated docs (02b5993)

• Merge branch 'main' of https://github.com/KennethEnevoldsen/scandinavian-embedding-benchmark into update-tables (db3ae97)

• Merge pull request #103 from KennethEnevoldsen/restructure_model_interface

Added LazyLoadEncoder, added SebModel and removed EmbeddingModel (4e1cd36)

• Fixed linting (82c0a49)

• Changed Norwegian Bitext Mining revision to None (a5675eb)

• Added docstring to LazyLoadEncoder, adjusted api.md to new interface. (c48e9cf)

• Changed model building to use SentenceTransformerWithTaskEncode in CLI (c85aad0)

• Ran linting on danish tasks (3663cd8)

• Ran linting (8f1c3b3)

• Removed verbose parameter, as it does not exist (2a743d0)

• Converted new OpenAI models to the new interface (8b24890)

• Fixed syntax error in Cohere (e1d18f1)

• Fixed type errors in speed task (834e5c7)

• Fixed tuple.extend type errors (341930a)

• Merge branch 'main' into restructure_model_interface (7b20806)

v0.11.0

v0.11.0 (2024-01-29)

Fix

• fix: Added cache for speed benchmark (f3cd21e)

Unknown

• Merge pull request #120 from KennethEnevoldsen/ran-speed-bench

fix: Added cache for speed benchmark (b4596c3)

• Merge pull request #109 from KennethEnevoldsen/add-sts-retrieval-dataset

Add SNL Clustering task (9d1dae4)

• Merge pull request #110 from KennethEnevoldsen/sts-retrieval

Add SNL retrieval (9412d71)

• Merge branch 'add-sts-retrieval-dataset' into sts-retrieval (a1216bf)

v0.10.0

v0.10.0 (2024-01-29)

Unknown

• Merge pull request #119 from KennethEnevoldsen/updated-coverage-tables

Updated coverage tables (94f72d5)

• Merge pull request #108 from KennethEnevoldsen/add-sts-retrieval

Restructured MTEB (115acef)

v0.9.4

v0.9.4 (2024-01-28)

Fix

• fix: speed benchmark actually runs the speed tasks (7ae3ef0)

v0.9.3

v0.9.3 (2024-01-28)

Fix

• fix: Added new OpenAI Models (e096ef5)