Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automatically generate readthedocs list of all plugins and their info #72

Closed
wants to merge 536 commits into from
Closed

Automatically generate readthedocs list of all plugins and their info #72

wants to merge 536 commits into from

Conversation

kvfairchild
Copy link
Contributor

@kvfairchild kvfairchild commented Sep 23, 2022 •
edited
Loading

Populates readthedocs documentation (plugins.rst) with information about all available plugins and a project bibliography. Relies on information, e.g. BiBTeX, provided in plugin __init__.py.

Adds the following dependencies:

Plugin info listing
image

Bibliography
image

benlipkin and others added 30 commits April 4, 2022 12:13
@benlipkin
added basic ceiling feature; interface integration open to adjustment
b2d75c1
@benlipkin
added neuroid split coord for rsa, cka, and other identity maps
ccf0b50
@gretatuckute
add sparsity tools
da0ff2b
@gretatuckute
initial code for model card (config info, variance explained, sparsity)
96287c0
@benlipkin
ceilings now have same dimensionality as scoring
d45eaa4
@aalok-sathe
rm unused script
b564b14
@aalok-sathe
add pereira18 to a script within benchmarks module
2fc2deb
@aalok-sathe
fix docstring for Dataset; remove redundant __init__ to help support …
b8345de 
…caching behavior
@aalok-sathe
remove repr() to fall back on default repr() from _Cacheable
2216018
@aalok-sathe
MWE implementation of caching and recovering from cache for _Cacheabl…
154f143 
…e-implementing classes
@aalok-sathe
add a "dataset_name" attribute on _Dataset for identifiability. defau…
5b3e374 
…lts to None
@aalok-sathe
add a test for caching behavior; currently tests Dataset
abc6432
@aalok-sathe
update lockfile and pyproject.toml
681359f
@aalok-sathe
Merge branch 'dev' of github.com:language-brainscore/lang-brainscore-…
52caa02 
…fuzzy-potato into dev
@aalok-sathe
fix how nan == nan is treated in xarrays in the _Cacheable.__eq__ method
fc64204
@aalok-sathe
add some docstrings to cache testing stuff
c72ff98
@aalok-sathe
clean old, unused directory
72b99da 
deleted:    langbrainscore/benchmark_packaging/.gitkeep
deleted:    langbrainscore/benchmark_packaging/package_benchmarks.py
@aalok-sathe
add yapf to dev deps
0cf9586
@aalok-sathe
write docstrings in xarray utils
b614cec
@aalok-sathe
simplify case method
cdb0920
@aalok-sathe
write an EncoderRepresentations dataclass to support caching of ANN…
961cef6 
… outputs
@aalok-sathe
add tuple to supported cacheable attributes
de5ef74
@aalok-sathe
move many of the params to __init__ so they become attrs of self and …
afb6f0c 
…are cached with `HugingFaceEncoder`
@aalok-sathe
add enforce for some runtime type checking
fba0c07
@gretatuckute
Update .gitignore
6ff6c9c
@gretatuckute
migrate encoder resources to separate script
afd93c6
@gretatuckute
update test script args
ffbeb34
@aalok-sathe
remove enforce
5a7a0d2
@aalok-sathe
kw_only kwarg is not valid before 3.10
bc19f2a
@aalok-sathe
add mtg notes
05f6b7b
@kvfairchild kvfairchild changed the title Automatically generate readthedocs plugins.rst with a list of all plugins and info Automatically generate readthedocs plugins.rst with a list of all plugins and their info Sep 25, 2022
@kvfairchild kvfairchild changed the title Automatically generate readthedocs plugins.rst with a list of all plugins and their info Automatically generate readthedocs list of all plugins and their info Sep 25, 2022
@kvfairchild kvfairchild linked an issue Sep 26, 2022 that may be closed by this pull request
display available {data,metrics,benchmarks,models} in documentation #57
Open
mschrimpf
mschrimpf reviewed Sep 27, 2022
View reviewed changes
Copy link
Member

@mschrimpf mschrimpf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool. The type (data, benchmarks etc) and instance (Pereira2018.language, wikitext-2/text etc) headings look fairly similar on the page, but that's a sphinx issue.

Is there a way to also get the class documentation in there? E.g. the Futrell2018 benchmark has documentation. The _Pereira2018 base class also has documentation, I think there's a way to tell the calling functions to copy the documentation

docs/source/index.rst Outdated
Comment on lines 26 to 27
modules/api_reference
modules/plugins
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
modules/api_reference
modules/plugins
modules/plugins
modules/api_reference

Copy link
Member

@mschrimpf mschrimpf Sep 27, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API reference at the bottom imo

docs/source/modules/plugins.rst Outdated
@@ -0,0 +1,106 @@

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so will this file be auto-generated by the readthedocs build? I.e. won't be part of the repository once that happens?

mschrimpf and others added 4 commits September 28, 2022 05:06
@mschrimpf
Usage examples (#69)
854f7a7 
* relax python version

* try exact python 3.7 version for colab

* add example scripts for scoring (separate layers), species comparisons

* fix human-human behavioral comparison

* fix model behavioral/neural comparisons

* undo python specification changes

* add human-human neural comparison

* progress bar over huggingface text digestion

* distilgpt2 score on Futrell, and glove on Pereira2018

* remove whitespace

* explain `[1:]`

* show examples in docs

* try with examples.md in source

* remove punctuation for embedding models

* fix colon removal
@kvfairchild
Option to specify single plugin test (or single test file) with loggi…
a02a21c 
…ng (#73)

* option to run single test

* typo

* cleanup

* documentation

Co-authored-by: Katherine Fairchild <[email protected]>
@mschrimpf
relax python version to >=3.7, test with python 3.7 through 3.9 (#74)
bc3b57b 
* relax python version to `>=3.7`

* test with python 3.7 through 3.9 (not testing python 3.10, travis doesn't have it yet)

* `pip list` to print versions

* use exact setuptools version in python 3.7

* fix travis.yml syntax
@mschrimpf
Merge branch 'main' into kvf/sphinx
5ea9adc
@mschrimpf
Copy link
Member

@kvfairchild what's the status of this PR?

@kvfairchild
Copy link
Contributor Author

@kvfairchild what's the status of this PR?

Deprioritized behind plugin environments.

@mschrimpf mschrimpf closed this Oct 26, 2022
@mschrimpf
Copy link
Member

mschrimpf commented Oct 26, 2022 •
edited
Loading

Sorry, this PR was closed after combining commit history. @kvfairchild can you please open a new PR? I'm afraid the simplest solution is to 1. checkout the branch locally, 2. copy all files into a separate location, 3. checkout main, 4. update main, 5. create a new branch, 6. copy all files back again overwriting the files in main, 7. push and open a PR.

@mschrimpf mschrimpf deleted the kvf/sphinx branch November 8, 2022 23:34
@kvfairchild kvfairchild restored the kvf/sphinx branch November 2, 2023 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mschrimpf mschrimpf mschrimpf left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

display available {data,metrics,benchmarks,models} in documentation
6 participants
@kvfairchild @mschrimpf @benlipkin @gretatuckute @aalok-sathe @d-val