notes updates

_blog/misc/24_tensor_product_repr.md

@@ -58,6 +58,6 @@ This example simplifies many aspects for clarity. In practice, the dimensions fo
 
 **Notes**
 - Learning in TPRs involves optimizing the filler and role vectors to optimize the reconstruction of input structures from their TPRs, achievable through gradient descent or other techniques
-- The simple outer product forms a strong foundation for symbolic learning and an ind
-- TPRs continue to be a major part of ongoing research, e.g. see [this paper](https://ojs.aaai.org/aimagazine/index.php/aimagazine/article/view/18599) for a forward-looking perspective or the [TP-Transformer](https://arxiv.org/abs/1910.06611) that enhances transformers with role vectors
+- The simple outer product forms a strong foundation for symbolic learning
+- TPRs remain a major area of active research, e.g. see [this paper](https://ojs.aaai.org/aimagazine/index.php/aimagazine/article/view/18599) for a forward-looking perspective or the [TP-Transformer](https://arxiv.org/abs/1910.06611) that enhances transformers with role vectors
 

_includes/00_about.html

@@ -21,7 +21,7 @@ <h1 class="brand-heading">&nbsp;</h1>
                         <ul class="list-inline social-buttons">
                             <li><a href="https://scholar.google.com/citations?user=XpttKK8AAAAJ&hl=en&oi=sra"><i
                                         class="fa fa-graduation-cap fa-fw"></i></a></li>
-                            <li><a href="mailto:[email protected]"><i class="fa fa-envelope fa-fw"></i></a>
+                            <li><a href="mailto:[email protected]"><i class="fa fa-envelope fa-fw"></i></a>
                             </li>
                             <li><a href="{{ site.url }}/assets/cv_chandan.pdf">cv</a></li>
                             <li><a href="https://github.com/csinva"><i class="fa fa-github fa-fw"></i></a></li>

_notes/ml/nlp.md

@@ -150,20 +150,14 @@ Nice repo keeping track of progress [here](https://github.com/sebastianruder/NLP
   - for full sentence, use markov assumption
 - multi-token decoding for classification - regular beam search will favor shorter results over longer ones on average since a negative log-probability is added at each step, yielding lower (more negative) scores for longer sentences
 
+- smoothing ngram models 
+
 ## topic modeling
 
 **topic models (e.g. LDA)** - apply unsupervised learning on large sets of text to learn sets of associated words
 
 - LDA = latent dirichlet allocation
 
-## interpretable prediction models
-
-- [Neural Bag-of-Ngrams](https://ojs.aaai.org/index.php/AAAI/article/view/10954) (li et al. 2017) - learn embedding vectors for ngrams via deep version of skip-gram
-- [Improving N-gram Language Models with Pre-trained Deep Transformer](https://arxiv.org/abs/1911.10235) (wang et al. 2019) - use transformer to generate synthetic data for new n-gram model (language model, doesn't extend to classification)
-  - [Improvements to N-gram Language Model Using Text Generated from Neural Language Model](https://ieeexplore.ieee.org/abstract/document/8683481?casa_token=7iD-YiGsHTAAAAAA:N3XmuRk27wGttURXYIYDbxdADVdhJMeUeBvVugq0EbyMst-zrm93wPZtc37uUBBtUPXKPrxvGZJC) (suzuki et al. 2019) - generate synthetic data from RNNs for new n-gram model
-- [fasttext](https://www.ijcai.org/Proceedings/16/Papers/401.pdf) (jin et al. 2016)
-- [(DirtyCat): Encoding High-Cardinality String Categorical Variables](https://ieeexplore.ieee.org/abstract/document/9086128) (cerda & varoquax, 2020) - use embedding model to improve string categorical variables
-
 ## grammar / parse-tree exraction
 
 *notes/figures from [nltk book ch 8/9](https://www.nltk.org/book/)*

_notes/neuro/comp_neuro.md

@@ -1011,6 +1011,7 @@ subtitle: Diverse notes on various topics in computational neuro, data-driven ne
 ## nlp
 
 - Mapping Brains with Language Models: A Survey ([Karamolegkou et al. 2023](https://arxiv.org/abs/2306.05126))
+- Semantic encoding during language comprehension at single-cell resolution ([jamali...fedorenko, williams, 2024](https://www.nature.com/articles/s41586-024-07643-2))
 - interpreting brain encoding models
   - [Brains and algorithms partially converge in natural language processing](https://www.nature.com/articles/s42003-022-03036-1#Sec9) (caucheteux & king, 2022)
     - best brain-mapping are obtained from the middle layers of DL models
@@ -1047,7 +1048,16 @@ subtitle: Diverse notes on various topics in computational neuro, data-driven ne
 - decoding models
   - [Semantic reconstruction of continuous language from non-invasive brain recordings](https://www.biorxiv.org/content/10.1101/2022.09.29.509744v1) (lebel, jain, & huth, 2022) - reconstruct continuous natural language from fMRI
     - [Decoding speech from non-invasive brain recordings](https://arxiv.org/abs/2208.12266) (defossez, caucheteux, ..., remi-king, 2022)
-  - The generalizability crisis ([yarkoni, 2020](https://mzettersten.github.io/assets/pdf/ManyBabies_BBS_commentary.pdf)) - there is widespread difficulty in converting informal verbal hypotheses into quantitative models
+- Bilingual language processing relies on shared semantic representations that are modulated by each language ([chen...klein, gallant, deniz, 2024](https://www.biorxiv.org/content/10.1101/2024.06.24.600505v1)) - shared semantic representations are modulated by each language
+
+## theories of explanation
+
+- The generalizability crisis ([yarkoni, 2020](https://mzettersten.github.io/assets/pdf/ManyBabies_BBS_commentary.pdf)) - there is widespread difficulty in converting informal verbal hypotheses into quantitative models
+- Formalising the role of behaviour in neuroscience ([piantadosi & gallistel, 2024](https://onlinelibrary.wiley.com/doi/10.1111/ejn.16372)) - can build isomorphisms between behavior and mathematical theories of representations
+- [NeuroSynth website](https://www.neurosynth.org/analyses/terms/)
+  - Large-scale automated synthesis of human functional neuroimaging data ([yarkoni, poldrack, nichols, van essen, & wager, 2011](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3146590/pdf/nihms-300972.pdf)) 
+  - NeuroQuery, comprehensive meta-analysis of human brain mapping ([dockes, poldrack, ..., yarkonig, suchanek, thirion, & varoquax](https://elifesciences.org/articles/53385)) [[website](https://neuroquery.org/query?text=checkerboard)]
+    - train on keywords to directly predict weights for each query-expanded keyword and the produce linearly combined brainmap
 
 ## vision
 
@@ -1058,6 +1068,7 @@ subtitle: Diverse notes on various topics in computational neuro, data-driven ne
 - A variational autoencoder provides novel, data-driven features that explain functional brain representations in a naturalistic navigation task ([cho, zhang, & gallant, 2023](https://jov.arvojournals.org/article.aspx?articleid=2792546))
 - What's the Opposite of a Face? Finding Shared Decodable Concepts and their Negations in the Brain ([efird...fyshe, 2024](https://arxiv.org/abs/2405.17663)) - build clustering shared across subjects in CLIP space
 
+
 # advanced topics
 
 ## high-dimensional (hyperdimensional) computing
@@ -1562,6 +1573,11 @@ the operations above allow for encoding many normal data structures into a singl
 
 - [Engineering a Less Artificial Intelligence](https://www.sciencedirect.com/science/article/pii/S0896627319307408) (sinz…tolias, 2019) - overview of ideas to make DNNs more brain-like
 
+## credit assignment
+
+- Backpropagation and the brain ([lillicrap...hinton, 2020](https://www.nature.com/articles/s41583-020-0277-3))
+- Inferring neural activity before plasticity as a foundation for learning beyond backpropagation ([song...bogacz, 2024](https://www.nature.com/articles/s41593-023-01514-1)) - use energy minimization before updating weights to help decide which weights to update
+
 ## biological constraints for DNNs
 
 - Aligning DNN with brain responses

_notes/research_ovws/ovw_interp.md

@@ -147,6 +147,7 @@ For an implementation of many of these models, see the python [imodels package](
 - why might these be useful?
   - [The Magical Mystery Four: How is Working Memory Capacity Limited, and Why?](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2864034/) (cowan, 2010) - a central memory store limited to 3 to 5 meaningful items in young adults
 
+- [Feldman (2000)](https://pubmed.ncbi.nlm.nih.gov/11034211/): humans can understand logical rules with boolean complexity of up to 5–9, depending on their ability, where the boolean complexity is the length of the shortest Boolean formula logically equivalent to the concept, usually expressed in terms of the number of literals
 - connections
   - every decision list is a (one-sided) decision tree
   - every decision tree can be expressed as an equivalent decision list (by listing each path to a leaf as a decision rule)