From 2f6ab371c49f26bd6b3283dabe929907c561ca23 Mon Sep 17 00:00:00 2001 From: kreeben Date: Sun, 7 Nov 2021 12:41:14 +0100 Subject: [PATCH] move todo list into readme --- README.md | 16 ++++++++++++++++ src/todo.txt | 12 ++++++++---- 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 4b8be4db..07f3bacf 100644 --- a/README.md +++ b/README.md @@ -67,3 +67,19 @@ Currently, Wikipedia size data sets produce indices capable of sub-second phrase - [ ] v2.1 - voice-to-text - [ ] v2.2 - text-to-image - [ ] v2.3 - AI + +## Backlog + +### Huge +- Distribute data set across many servers (sharding, replication; RPC) or in other ways allow for horisontal scaling + +### Big +- Memory mapping (to increase speed of querying and perhaps also writing; to increase scalability) +- Update index (allow removal of documents; allow appending to an already persisted index token's postings list) +- Async IO (for scalability) +- Indexing of types other than string +- Enable combining fields with different types in a document/model +- Split application into "crawler" and "search" + +### Small +- Make loggers non-mandatory ctor params \ No newline at end of file diff --git a/src/todo.txt b/src/todo.txt index aba0ec25..97624aa3 100644 --- a/src/todo.txt +++ b/src/todo.txt @@ -1,11 +1,15 @@ +huge +---- +distribute data set across many servers (sharding, replication; RPC) or in other ways allow for horisontal scaling + big ---- -memory mapping -update index -refactor indexing (one word occurs in one page only) +memory mapping (to increase speed of querying and perhaps also writing; to increase scalability) +update index (allow removal of documents; allow appending to an already persisted index token's postings list) async IO (for scalability) indexing of types other than string -enable combining types in a document/model +enable combining fields with different types in a document/model +split application into "crawler" and "search" small -----