Skip to content

Releases: brain-score/vision

v2.2.4

17 Dec 21:07
@github-actions github-actions
v2.2.4
c168d36
Compare
Choose a tag to compare
v2.2.4 Latest
Latest

Full Changelog: v2.2.4...v2.2.4

Assets 2
Loading

v2.2.3

17 Dec 19:38
@github-actions github-actions
v2.2.3
506952b
Compare
Choose a tag to compare
v2.2.3

What's Changed

Read more

Contributors

KartikP, kvfairchild, and 7 other contributors
Assets 2
Loading

v2.2.2

26 Nov 19:51
@github-actions github-actions
v2.2.2
ad8f594
Compare
Choose a tag to compare
v2.2.2

What's Changed

Read more

Contributors

KartikP, kvfairchild, and 6 other contributors
Assets 2
Loading

v2.2.1

13 Nov 21:53
@github-actions github-actions
v2.2.1
2ff306d
Compare
Choose a tag to compare
v2.2.1

What's Changed

New Contributors

Full Changelog: v2.1.0...v2.2.1

Contributors

KartikP, kvfairchild, and 5 other contributors
Assets 2
Loading

v2.2.0

13 Nov 16:35
@github-actions github-actions
v2.2.0
cb28c9a
Compare
Choose a tag to compare
v2.2.0

What's Changed

New Contributors

Full Changelog: v2.1.0...v2.2.0

Contributors

KartikP, kvfairchild, and 5 other contributors
Assets 2
Loading

v2.1.0

11 Sep 14:31
@mike-ferguson mike-ferguson
v2.1.0
4e0ca0c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v2.1.0

What's changed

Version 2.1.0 brings much-requested support for Python 3.11, finally allowing users to use packages and models that require Python >=3.8. Note: Brain-Score is officially deprecating Python <3.8 - this means that as of Version 2.1.0, Brain-Score will no longer work with Python <3.8.

Specifically:

  1. Added support for Python >=3.8, up to and including 3.11, across vision and core. This allows users to score models that previously needed >3.7, as well as use any packages that needed >3.7 as well.
  2. Retired support for Keras and Tensorflow models - going forward, Brain-Score will only support Pytorch models.

Looking Ahead

The Brain-Score Team is currently working on the following:

  1. Overhauled scoring system: users will soon be able to score models via AWS EC2 instances instead of our local MIT cluster, resulting in a significantly faster and more reliable scoring experience.
  2. Refreshed and updated material for model tutorials, including bug fixes and better documentation for new users, as well as a clearer example of how to score a custom model.
  3. Adding 75 models to the vision/models folder, to allow users to instantly download and score any of the top 100 models.
  4. The much-requested feature of allowing users to instantly download the leaderboard data in the form of a CSV file from the website.

Full Changelog: v2.0.0...v2.1.0

Assets 2
Loading

v2.0.0

19 Aug 19:52
@mike-ferguson mike-ferguson
v2.0.0
cda897b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v2.0.0

What's changed

We are thrilled to introduce Brain-Score 2.0, a major upgrade to the Brain-Score platform that empowers researchers, developers, and enthusiasts in the fields of neuroscience, cognitive science, and artificial intelligence to directly compare computational models of the brain with human and non-human primate behavior and brain recordings.

Here are the Brain-Score 2.0 features we are most excited about:

  1. Flexibility: new plugin system for easier contribution and collaboration: Brain-Score 2.0 introduces a modular plugin system that makes it easy to develop, combine, and contribute data, metrics, benchmarks, and models. Researchers can now more easily submit their own components and stress-test leading models or submit their own state-of-the-art model.
  2. Accessibility: enhanced website and documentation. Brain-Score’s web presence got a facelift. Our revamped website provides an improved user experience with more intuitive navigation, a simplified leaderboard, and detailed tutorials. We believe that making the platform easier to use is crucial for fostering collaboration and innovation.
  3. Language: broadening to a new domain. Brain-Score 2.0 introduces Brain-Score Language, which extends the Brain-Score platform from modeling the visual ventral stream to the human language system. The addition of Brain-Score Language reflects our philosophy that the Brain-Score approach to model benchmarking is fundamentally domain agnostic, and represents a step toward a future multimodal Brain-Score.

There are other quality of life changes that we hope will contribute to the platform's ease of use and sense of community, including:

  1. A centralized organization repo with clear purpose descriptions.
  2. A domain-independent, simplified way of scoring models, as seen here.
  3. Better error reporting for scoring runs, as well as automated Github merge functionality for user contributions via pull requests.
  4. A centralized repo of models to be easily accessible and runnable by any user.
  5. Enhanced FAQ and troubleshooting with the most common errors encountered, as well as their solutions.
  6. A dedicated Slack channel for the community, that is always accepting new members!

...and much more!

Looking Ahead

As Brain-Score grows, we hope that regular version updates and patch notes will be released on a bi-weekly schedule (once every two weeks), and major feature additions will be announced as they are completed. Normally these patch notes would include direct links to PRs or issues, but as the 2.0.0 update was so large, those are omitted in this release notes announcement.

The Brain-Score team has even more features and fixes on the horizon, including:

  1. Switching to a more stable and faster compute cluster in order to produce model scores in a reliable and efficient way. This should result in significantly less scoring errors and downtime.
  2. Updating all our code to support Python 3.11. This is one of our most popular requests, and we hope to have that done by the next release (2.1.0)
  3. Further improved user documentation, with clearer setup and troubleshooting for scoring models (including custom models).
  4. Adding new models to the model repos (we hope to have the top 100 models all included on Github within the next few releases).
  5. Other quality-of-life changes, including adding an option for downloading the leaderboard as a CSV file

Full Changelog: v1.3...v2.0.0

Assets 2
Loading

v1.3

03 Aug 19:19
@mschrimpf mschrimpf
v1.3
5e43c94
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.3

What's changed

New Contributors

Full Changelog: v1.2...v1.3

Contributors

mschrimpf, dmayo, and 2 other contributors
Assets 2
Loading

v1.2

27 Jun 15:07
@mschrimpf mschrimpf
v1.2
919708b
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.2

Release for 2022 Brain-Score competition (http://www.brain-score.org/competition/)

High-level changes

Full Changelog: v1.1...v1.2

Contributors

stothe2, mschrimpf, and 4 other contributors
Assets 2
Loading

v1.1

27 Jun 15:06
@mschrimpf mschrimpf
v1.1
c0cbeed
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.1

Release of V1 property benchmarks (https://www.biorxiv.org/content/10.1101/2021.03.01.433495)

What's Changed

Full Changelog: v1.0...v1.1

Contributors

mschrimpf, dapello, and 3 other contributors
Assets 2
Loading