Search facility

[lmq-10]

Hello Prot (and Denote community as a whole). I have been using and enjoying this fantastic note-taking tool for a while. Eventually, as my collection of notes grew, I needed a search facility, and not content with the available options, I went and developed my own. Today I published the code and wrote documentation for it just in case someone finds it useful, too.

• https://github.com/lmq-10/denote-search

I think this feature is not planned (judging by #71), specially now that consult-denote exists, but there it is anyway!

[protesilaos]

Thank you for doing this! Do you want to prepare a brief description for our README here? We already mention other packages and I am happy to include yours there as well.

[lmq-10]

Oh, sure! Maybe something like this would work:

---

denote-search provides a search utility for Denote. It allows you to search for a regular expression in the content of your notes. Its main advantages over other similar tools are the possibility of filtering the results by file name and doing further searches in the files matched previously. This allows for advanced usage (think about finding a note with two or three specific words in different lines and with a specific keyword). More features are described in its comprehensive manual. denote-search builds upon standard Emacs libraries, namely Xref, and so it doesn't have external dependencies other than Denote itself.

---

Note that I haven't published the package to any ELPA yet. I wanted to see if there was any interest first. Also, I preferably would like to have it in GNU ELPA, but I still have to figure out the procedure.

[protesilaos]

About ELPA, you will need to assign copyright to the Free Software Foundation and then email the emacs-devel mailing list to ask for the inclusion of your package on ELPA. I have included the copyright assignment form in the manual of Denote. Evaluate this:

(info "(denote) Contributing")

Actually, you need this even for code that you submit to Denote.

---

About your package, I think what you are doing is useful. The question I have is if it makes sense to have it as part of the main Denote package or to keep it separate. The reason I am asking this is because we already have denote-sort-dired which is a search interface of sorts for file names. Maybe the current arrangement is fine, but I am happy to have it some other way if you think it makes sense.

[lmq-10]

Thank you for the information regarding copyright assignment.

About your question, I guess you'll know better. I tried to write denote-search in a way that would allow it to become part of Denote without issues, as an optional module if anything; it's pretty self-contained, has no dependencies, and its codebase is small. Naturally, I'd be very happy to see my little package integrated here, but again, that's up to you. I still have to assign copyright to the FSF though, but I plan to do it anyway.

(Oh, now that you mention denote-sort-dired, I realized that's a perfect way to start a search in a predefined set of files! I'll add a function to search in marked dired files to my package, it should be trivial. That way it will be better integrated, too.)

[protesilaos]

From: Lucas Quintana ***@***.***> Date: Tue, 31 Dec 2024 11:41:34 -0800 Thank you for the information regarding copyright assignment.

You are welcome!

About your question, I guess you'll know better. I tried to write `denote-search` in a way that would allow it to become part of Denote without issues, as an optional module if anything; it's pretty self-contained, has no dependencies, and its codebase is small. Naturally, I'd be very happy to see my little package integrated here, but again, that's up to you. I still have to assign copyright to the FSF though, but I plan to do it anyway.

It is also an open question for me: what do we all think is better in terms of organising the code we have. I have been adding these optional extensions to the main repo (like denote-sort.el, the Org dynamic blocks, and the new denote-sequence.el), but maybe they should all be their own little packages. The upside with having them all together is that they are more discoverable. The downside is that someone may get the impression the package is "bloated" because it does too many things. In terms of maintenance though, the workload is the same from my perspective as I would be maintaining the standalone packages as well. If we merge your code here, then I will grant you write access to the repo. I should have done the same with @jeanphilippegg, whose contributions I always merge. Alternatively, we split up the code to many smaller packages and take care to document the available options.

(Oh, now that you mention `denote-sort-dired`, I realized that's a perfect way to start a search in a predefined set of files! I'll add a function to search in marked dired files to my package, it should be trivial. That way it will be better integrated, too.)

Very well! Maybe we would want denote-sort-dired (or something like it) to eventually be part of your search facility rather than be in denote-sort.el. I think it makes sense conceptually.

…

-- Protesilaos Stavrou https://protesilaos.com

[lmq-10]

It is also an open question for me: what do we all think is better in terms of organising the code we have.

Well, it is certainly not an easy question. If you are looking for opinions, I think it is fine to keep extensions in the main repo. It comes to my mind how vertico manages it, with extensions grouped there (in the extensions directory) for anyone who wants extra functionality, but nevertheless keeping the main vertico.el as simple as possible. I don't think anyone believes vertico is bloated; if anything, it is a top quality program and people praise it as such. So, I'd take it as an example of good code practices, but again, this is just my opinion.

If we merge your code here, then I will grant you write access to the repo.

If you think that's the right thing to do, then great! But just for you to know, making pull requests doesn't bother me. I don't think denote-search will require too much maintenance, unless I start adding new features or something like that.

Very well! Maybe we would want denote-sort-dired (or something like it) to eventually be part of your search facility rather than be in denote-sort.el. I think it makes sense conceptually.

That could work. However, as I see it, they still cover different use-cases. denote-sort-dired produces a (sorted) list of files matching a regular expression. That indeed is useful for the purposes of denote-search, because it allows it to search in that subset of files, but it is also useful for many other things: copying these files, moving them, renaming them, merely looking at them, et cetera. Similarly, while a fixed set of files as the one created by denote-sort-dired can be used as a starting point for denote-search, there are still other ways of giving it a list of files. Now that it can search on marked dired files, any dired buffer can serve as a source of files, not only those of denote-sort-dired. I just pushed a change that allows it to search on the files referenced in a region, too, which I think is even more useful because it works outside dired (for a list of linked files in a note, for instance). And it still has filtering built-in, so a set of initial files is not even required.

So, I think they are better separated. They are still composable anyway, so they can be used in tandem if desired.

[protesilaos]

From: Lucas Quintana ***@***.***> Date: Wed, 1 Jan 2025 09:18:49 -0800 > It is also an open question for me: what do we all think is better in terms of organising the code we have. Well, it is certainly not an easy question. If you are looking for opinions, I think it is fine to keep extensions in the main repo. It comes to my mind how [vertico](https://github.com/minad/vertico) manages it, with extensions grouped there (in the `extensions` directory) for anyone who wants extra functionality, but nevertheless keeping the main `vertico.el` as simple as possible. I don't think anyone believes `vertico` is bloated; if anything, it is a top quality program and people praise it as such. So, I'd take it as an example of good code practices, but again, this is just my opinion.

Yes, this is a good approach as well. I do not think it is bloated either. Just saying. Does putting the files in a subdirectory create any issues with how they are loaded? I have a vague memory of a user not being able to load the Vertico extensions because of the load-path, but I forgot the kind of setup they were running.

> If we merge your code here, then I will grant you write access to the repo. If you think that's the right thing to do, then great! But just for you to know, making pull requests doesn't bother me. I don't think `denote-search` will require too much maintenance, unless I start adding new features or something like that. > Very well! Maybe we would want denote-sort-dired (or something like > it) to eventually be part of your search facility rather than be in > denote-sort.el. I think it makes sense conceptually. That could work. However, as I see it, they still cover different use-cases. `denote-sort-dired` produces a (sorted) list of files matching a regular expression. That indeed is useful for the purposes of `denote-search`, because it allows it to search in that subset of files, but it is also useful for many other things: copying these files, moving them, renaming them, merely looking at them, et cetera. Similarly, while a fixed set of files as the one created by `denote-sort-dired` can be used as a starting point for `denote-search`, there are still other ways of giving it a list of files. Now that it can search on marked dired files, any dired buffer can serve as a source of files, not only those of `denote-sort-dired`. I just pushed a change that allows it to search on the files referenced in a region, too, which I think is even more useful because it works outside dired (for a list of linked files in a note, for instance). And it still has filtering built-in, so a set of initial files is not even required. So, I think they are better separated. They are still composable anyway, so they can be used in tandem if desired.

Thank you for the feedback! I say we keep everything as-is for the time being. This discussion has been my long way of saying "I am open to ideas about how best to organise Denote and that the fact we have not had these discussions does not imply otherwise". Let me know if you need any help with the GNU ELPA submission. I am happy to answer any questions you may have.

…

-- Protesilaos Stavrou https://protesilaos.com

[lmq-10]

Does putting the files in a subdirectory create any issues with how they are loaded?

It does if the user does not install the package from the package archive. For instance, when installing manually, one has to explicitly add the extensions directory to the load path. That's probably needed when using package managers such as straight, too, though I never used those. I'm not sure if it's an "issue", but it could be considered unexpected behaviour to someone not familiar with load paths.

(By the way, I wasn't really suggesting to use an extensions directory for Denote, it was just an example of a package including extensions in the main repo. consult, by the same author, has files with extra functionality together with consult.el)

I say we keep everything as-is for the time being.

Sounds good to me!

Let me know if you need any help with the GNU ELPA submission. I am happy to answer any questions you may have.

I think I'm good for now. I'll proceed with the copyright assignment formality. If I have any questions after that, I'll reach you by e-mail (if that doesn't bother you, that is).