update notes

_notes/neuro/comp_neuro.md

@@ -898,7 +898,7 @@ subtitle: Diverse notes on various topics in computational neuro, data-driven ne
 - tms
   - genetically-targeted tms: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4846560/
 - [ect - Electroconvulsive Therapy](https://www.psychiatry.org/patients-families/ect#:~:text=Learn%20about%20Electroconvulsive%2C%20therapy,the%20patient%20is%20under%20anesthesia.)  (sometimes also called electroshock therapy)
-  - [Identifying Recipients of Electroconvulsive Therapy: Data From Privately Insured Americans - PMC](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6248332/) - 100k ppl per year
+  - Identifying Recipients of Electroconvulsive Therapy: Data From Privately Insured Americans ([wilkinon...roenheck, 2018](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6248332/)) - 100k ppl per year
   - can differ in its application in three ways
     - electrode placement
       - used to be bilateral, now unilateral is more popular
@@ -913,6 +913,7 @@ subtitle: Diverse notes on various topics in computational neuro, data-driven ne
       - increased hippocampal neurogenesis and synaptogenesis
     - Electroconvulsive therapy: How modern techniques improve patient outcomes ([tirmizi, 2012)](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4193538/pdf/nihms497537.pdf)
     - The neurobiological effects of electroconvulsive therapy studied through magnetic resonance – what have we learnt and where do we go? ([ousdal et al. 2022](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8630079/pdf/nihms-1710166.pdf))
+    - Clinical EEG slowing induced by electroconvulsive therapy is better described by increased frontal aperiodic activity ([mith...soltani, 2023](https://www.nature.com/articles/s41398-023-02634-9))
 - local microstimulation with invasive electrodes
 
 
@@ -1374,6 +1375,8 @@ the operations above allow for encoding many normal data structures into a singl
 
   - The DeepTune framework for modeling and characterizing neurons in visual cortex area V4 ([abbasi-asl, ..., yu, 2018](https://www.biorxiv.org/content/10.1101/465534v1.abstract))
 
+  - Compact deep neural network models of visual cortex ([cowley, stan, pillow, & smith, 2023](https://www.biorxiv.org/content/10.1101/2023.11.22.568315v1.abstract))
+
 - XDream: Finding preferred stimuli for visual neurons using generative networks and gradient-free optimization ([2020](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1007973))
 - CORNN: Convex optimization of recurrent neural networks for rapid inference of neural dynamics ([dinc...tanaka, 2023](https://arxiv.org/abs/2311.10200)) - mouse population control
 
@@ -1505,18 +1508,18 @@ the operations above allow for encoding many normal data structures into a singl
 
 - https://xcorr.net/2023/01/01/2022-in-review-neuroai-comes-of-age/
 
-- [Neuroscience-Inspired Artificial Intelligence](https://www.cell.com/neuron/pdf/S0896-6273(17)30509-3.pdf) (hassabis et al. 2017)
+- Neuroscience-Inspired Artificial Intelligence ([hassabis et al. 2017](https://www.cell.com/neuron/pdf/S0896-6273(17)30509-3.pdf))
 
-- [Toward next-generation artificial intelligence: catalyzing the NeuroAI revolution](https://arxiv.org/abs/2210.08340) (zador, ...bengio, dicarlo, lecun, ...sejnowski, tsao, 2022)
+- Toward next-generation artificial intelligence: catalyzing the NeuroAI revolution ([zador, ...bengio, dicarlo, lecun, ...sejnowski, tsao, 2022](https://arxiv.org/abs/2210.08340))
 
 - Computational language modeling and the promise of in silico experimentation ([jain, vo, wehbe, & huth, 2023](https://direct.mit.edu/nol/article/doi/10.1162/nol_a_00101/114613/Computational-language-modeling-and-the-promise-of)) - 4 experimental design examples
 
-  - compare concrete & abstract words [(binder et al. 2005](https://direct.mit.edu/jocn/article-abstract/17/6/905/4017/Distinct-Brain-Systems-for-Processing-Concrete-and))
+  - compare concrete & abstract words ([binder et al. 2005](https://direct.mit.edu/jocn/article-abstract/17/6/905/4017/Distinct-Brain-Systems-for-Processing-Concrete-and))
   - contrast-based study of composition in 2-word phrase ([Bemis & Pylkkanen, 2011](https://www.jneurosci.org/content/31/8/2801.short))
   - checks for effects between group and individual ([lerner et al. 2011](https://www.jneurosci.org/content/31/8/2906))
   - forgetting behavior using controlled manipulations ([chien & honey, 2020](https://www.sciencedirect.com/science/article/pii/S0896627320301367))
 
-- [Dissociating language and thought in large language models](https://arxiv.org/abs/2301.06627) (mahowald, ..., tenebaum, fedorenko, 2023)
+- Dissociating language and thought in large language models ([mahowald, ..., tenebaum, fedorenko, 2023](https://arxiv.org/abs/2301.06627))
 
   - 2 competences
 
@@ -1539,10 +1542,12 @@ the operations above allow for encoding many normal data structures into a singl
 
     - modularity, curated data / diverse objectives, new benchmarks
 
-- [Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systems](https://ojs.aaai.org/index.php/aimagazine/article/view/18599) (smolensky, ..., gao, 2022)
+- Neurocompositional computing: From the Central Paradox of Cognition to a new generation of AI systems ([smolensky, ..., gao, 2022](https://ojs.aaai.org/index.php/aimagazine/article/view/18599))
 
 - [Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networks](https://www.semanticscholar.org/paper/Towards-NeuroAI%3A-Introducing-Neuronal-Diversity-Fan-Li/c0aae24f2e250c7d4b5aab608622dbb933f43a4d) (2023)
 
+- A rubric for human-like agents andNeuroAI ([momennejad, 2022](https://royalsocietypublishing.org/doi/epdf/10.1098/rstb.2021.0446)): 3 axes - human-like behavior, neural plausibility, & engineering
+
 - [Designing Ecosystems of Intelligence from First Principles](https://www.semanticscholar.org/paper/Designing-Ecosystems-of-Intelligence-from-First-Friston-Ramstead/98fcb39694d628788b555932f96134280f6a008e) (friston et al. 2022)
 
 - [NeuroAI - A strategic opportunity for Norway and Europe](https://www.semanticscholar.org/paper/NeuroAI-A-strategic-opportunity-for-Norway-and-Nichele-Sæbø/b5e7bacfdd6d080fce402a27b36757f6246eef4d) (2022)

_notes/research_ovws/ovw_transformers.md

@@ -195,7 +195,7 @@ See related papers in the [📌 interpretability](https://csinva.io/notes/resear
   - Self-Refine: Iterative Refinement with Self-Feedback ([madaan, ..., clark, 2023](https://arxiv.org/abs/2303.17651))
   - Self-Verification Improves Few-Shot Clinical Information Extraction ([gero et al. 2023](https://arxiv.org/abs/2306.00024))
   - SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models ([manakul...gales, 2023](https://arxiv.org/abs/2303.08896))
-- ACT-1: Transformer for Actions ([2022, Adept](https://www.adept.ai/act)) - transformer directly interacts with computer
+- ACT-1: Transformer for Actions ([2022, adept](https://www.adept.ai/act)) - transformer directly interacts with computer
 - ReAct: Synergizing Reasoning and Acting in Language Models ([yao...cao, 2022](https://arxiv.org/abs/2210.03629)) - use LLMs to generate reasoning traces + task-specific actions in interleaved manner
 
 # prompting
@@ -500,6 +500,7 @@ See related papers in the [📌 interpretability](https://csinva.io/notes/resear
   - Teach Llamas to Talk: Recent Progress in Instruction Tuning ([gao blogpost 2023](https://gaotianyu.xyz/blog/2023/11/30/instruction-tuning/))
 
   - Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs ([zhang et al. 2023](https://arxiv.org/abs/2311.02262))
+  - The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction ([sharma...misra, 2023](https://arxiv.org/abs/2312.13558))
   - human feedback
     - Learning to summarize with human feedback ([OpenAI, 2020](https://proceedings.neurips.cc/paper/2020/hash/1f89885d556929e98d3ef9b86448f951-Abstract.html))
     - Can language models learn from explanations in context? ([lampinen et al. 2022](https://arxiv.org/abs/2204.02329))
@@ -683,6 +684,8 @@ See related papers in the [📌 interpretability](https://csinva.io/notes/resear
 - Tree Transformer: Integrating Tree Structures into Self-Attention ([wang, .., chen, 2019](https://arxiv.org/pdf/1909.06639.pdf))
 - Waveformer: Linear-Time Attention with Forward and Backward Wavelet Transform ([zhuang...shang, 2022](https://arxiv.org/abs/2210.01989))
 - state space models (good overview in [albert gu thesis](https://searchworks.stanford.edu/view/14784021), 2023)
+  - mamba ([gu & dao, 2023](https://arxiv.org/abs/2312.00752))
+
 
 ## model merging / mixture of experts (MoE) / routing
 
@@ -785,8 +788,9 @@ mixture of experts models have become popular because of the need for (1) fast s
 
 ## embeddings
 
-- Instructor: One Embedder, Any Task: Instruction-Finetuned Text Embeddings ([su, ..., smith, zettlemoyer, yu, 2022](https://instructor-embedding.github.io)) - embedding is contextualized to eaach task
-- Text Embeddings Reveal (Almost) As Much As Text ([2023](https://openreview.net/pdf?id=wK7wUdiM5g0))
+- Instructor: One Embedder, Any Task: Instruction-Finetuned Text Embeddings ([su, ..., smith, zettlemoyer, yu, 2022](https://instructor-embedding.github.io)) - embedding is contextualized to each task
+- Text Embeddings Reveal (Almost) As Much As Text ([morris et al. 2023](https://arxiv.org/abs/2310.06816))
+- Uncovering Meanings of Embeddings via Partial Orthogonality ([jiang, aragam, & veitch, 2023](https://arxiv.org/abs/2310.17611))
 - Explaining embeddings
   - Computer-vision focused
     - Axiomatic Explanations for Visual Search, Retrieval, and Similarity Learning ([hamilton, lundberg…freeman, 2021](https://arxiv.org/abs/2103.00370)) - add in “second-order” methods that look at similarities between different image features in the 2 images being compared