Skip to content

Commit

Permalink
move todo list into readme
Browse files Browse the repository at this point in the history
  • Loading branch information
kreeben committed Nov 7, 2021
1 parent 5055cdb commit 2f6ab37
Show file tree
Hide file tree
Showing 2 changed files with 24 additions and 4 deletions.
16 changes: 16 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,3 +67,19 @@ Currently, Wikipedia size data sets produce indices capable of sub-second phrase
- [ ] v2.1 - voice-to-text
- [ ] v2.2 - text-to-image
- [ ] v2.3 - AI

## Backlog

### Huge
- Distribute data set across many servers (sharding, replication; RPC) or in other ways allow for horisontal scaling

### Big
- Memory mapping (to increase speed of querying and perhaps also writing; to increase scalability)
- Update index (allow removal of documents; allow appending to an already persisted index token's postings list)
- Async IO (for scalability)
- Indexing of types other than string
- Enable combining fields with different types in a document/model
- Split application into "crawler" and "search"

### Small
- Make loggers non-mandatory ctor params
12 changes: 8 additions & 4 deletions src/todo.txt
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
huge
----
distribute data set across many servers (sharding, replication; RPC) or in other ways allow for horisontal scaling

big
----
memory mapping
update index
refactor indexing (one word occurs in one page only)
memory mapping (to increase speed of querying and perhaps also writing; to increase scalability)
update index (allow removal of documents; allow appending to an already persisted index token's postings list)
async IO (for scalability)
indexing of types other than string
enable combining types in a document/model
enable combining fields with different types in a document/model
split application into "crawler" and "search"

small
-----
Expand Down

0 comments on commit 2f6ab37

Please sign in to comment.