Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[REVIEW]: JustSayIt.jl: A Fresh Approach to Open Source Voice Assistant Development #121

Open
21 of 42 tasks
whedon opened this issue Jan 9, 2023 · 55 comments
Open
21 of 42 tasks

[REVIEW]: JustSayIt.jl: A Fresh Approach to Open Source Voice Assistant Development #121

whedon opened this issue Jan 9, 2023 · 55 comments
Assignees
@matbesancon
Labels
accepted Julia published recommend-accept review

Comments

@whedon
Copy link
Collaborator

whedon commented Jan 9, 2023 •
edited by matbesancon
Loading

Submitting author: @omlins (Samuel Omlin)
Repository: https://github.com/omlins/JustSayIt.jl
Branch with paper.md (empty if default branch): JuliaConProceeding2022
Version: 0.3.0
Editor: @matbesancon
Reviewers: @PetrKryslUCSD, @pfitzseb
Archive: 10.5281/zenodo.14057584

Status

status

Status badge code:

HTML: <a href="https://proceedings.juliacon.org/papers/3325b7091de2a68c2ef6201768f1de79"><img src="https://proceedings.juliacon.org/papers/3325b7091de2a68c2ef6201768f1de79/status.svg"></a>
Markdown: [![status](https://proceedings.juliacon.org/papers/3325b7091de2a68c2ef6201768f1de79/status.svg)](https://proceedings.juliacon.org/papers/3325b7091de2a68c2ef6201768f1de79)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@PetrKryslUCSD & @pfitzseb, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @carstenbauer know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @PetrKryslUCSD

Conflict of interest

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Authorship: Has the submitting author (@omlins) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Paper format

  • Authors: Does the paper.tex file include a list of authors with their affiliations?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
  • Page limit: Is the page limit for extended abstracts respected by the submitted document?

Content

  • Context: is the scientific context motivating the work correctly presented?
  • Methodology: is the approach taken in the work justified, presented with enough details and reference to reproduce it?
  • Results: are the results presented and compared to approaches with similar goals?

Review checklist for @pfitzseb

Conflict of interest

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Authorship: Has the submitting author (@omlins) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Paper format

  • Authors: Does the paper.tex file include a list of authors with their affiliations?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
  • Page limit: Is the page limit for extended abstracts respected by the submitted document?

Content

  • Context: is the scientific context motivating the work correctly presented?
  • Methodology: is the approach taken in the work justified, presented with enough details and reference to reproduce it?
  • Results: are the results presented and compared to approaches with similar goals?
@whedon whedon mentioned this issue Jan 9, 2023
Closed
@whedon
Copy link
Collaborator Author

whedon commented Jan 9, 2023

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @PetrKryslUCSD, @pfitzseb it looks like you're currently assigned to review this paper 🎉.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/JuliaCon/proceedings-review) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/JuliaCon/proceedings-review:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

@whedon
Copy link
Collaborator Author

whedon commented Jan 9, 2023 

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.04 s (976.8 files/s, 83311.5 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Julia                           35            405            167           2756
YAML                             3              1              6            116
TOML                             2              4              0             29
Markdown                         1              4              0              9
-------------------------------------------------------------------------------
SUM:                            41            414            173           2910
-------------------------------------------------------------------------------


Statistical information for the repository '9ee0f315c44c9ac759829a5d' was
gathered on 2023/01/09.
No commited files with the specified extensions were found.

@whedon
Copy link
Collaborator Author

whedon commented Jan 9, 2023

PDF failed to compile for issue #121 with the following error:

 Can't find any papers to compile :-(

@carstenbauer
Copy link
Member

@whedon generate pdf from branch JuliaConProceeding2022

@whedon
Copy link
Collaborator Author

whedon commented Jan 9, 2023 

Attempting PDF compilation from custom branch JuliaConProceeding2022. Reticulating splines etc...

@whedon
Copy link
Collaborator Author

whedon commented Jan 9, 2023

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

@whedon
Copy link
Collaborator Author

whedon commented Jan 23, 2023

👋 @pfitzseb, please update us on how your review is going (this is an automated reminder).

@whedon
Copy link
Collaborator Author

whedon commented Jan 23, 2023

👋 @PetrKryslUCSD, please update us on how your review is going (this is an automated reminder).

This was referenced Jan 30, 2023
Closed
Closed
@pfitzseb
Copy link
Collaborator

Hey, some feedback on this submission:

  • In Fig 2, a using JustSayIt.API is required (same goes for the documentation at https://omlins.github.io/JustSayIt.jl/stable/usage/

  • Usage example fails due to incorrect type checks omlins/JustSayIt.jl#87 causes the example in Fig 2 to be non-functional.

  • Keyboard.type fails with "insecure command argument recognition" omlins/JustSayIt.jl#88 causes the Keyboard.type command to be non-functional.

  • I'm seeing worse latency than shown in the article, but I'm happy to chalk that up to benchmarking on a signigicantly less powerful machine.

    julia> versioninfo()
Julia Version 1.8.3
Commit 0434deb161e (2022-11-14 20:14 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, skylake)
  Threads: 1 on 8 virtual cores

julia> start(commands=commands, max_speed_subset=["ma", "select", "okay", "middle", "right", "double", "triple", "copy", "upwards", "downwards", "take"])
[ Info: JustSayIt: I am initializing...
[ Info: Listening for commands in English (United States) (say "sleep JustSayIt" to put me to sleep; press CTRL+c to terminate)...
[ Info: Starting command: click_left (latency: 34 ms)
[ Info: Starting command: click_left (latency: 8 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 25 ms)
[ Info: Starting command: click_left (latency: 23 ms)
[ Info: Starting command: click_left (latency: 32 ms)
[ Info: Starting command: click_left (latency: 30 ms)
[ Info: Starting command: click_left (latency: 25 ms)
[ Info: Starting command: click_left (latency: 21 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 23 ms)
[ Info: Starting command: click_left (latency: 27 ms)
[ Info: Starting command: click_left (latency: 20 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left
[ Info: Starting command: click_left (latency: 34 ms)
[ Info: Starting command: click_left (latency: 24 ms)
[ Info: Starting command: click_left (latency: 29 ms)
[ Info: Starting command: click_left (latency: 26 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 23 ms)
[ Info: Starting command: click_left (latency: 20 ms)
[ Info: Starting command: click_left (latency: 23 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 21 ms)
[ Info: Starting command: click_left (latency: 21 ms)
[ Info: Starting command: click_left (latency: 26 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 25 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 27 ms)
[ Info: Starting command: click_left (latency: 30 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 25 ms)
[ Info: Starting command: click_left (latency: 24 ms)
[ Info: Starting command: click_left (latency: 32 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 27 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 21 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 21 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 25 ms)
[ Info: Starting command: click_left (latency: 23 ms)
[ Info: Starting command: click_left (latency: 29 ms)
[ Info: Starting command: click_left (latency: 24 ms)
[ Info: Starting command: click_left (latency: 21 ms)
[ Info: Starting command: click_left (latency: 26 ms)
[ Info: Starting command: click_left (latency: 26 ms)
[ Info: Starting command: click_left (latency: 23 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 28 ms)
[ Info: Starting command: click_left (latency: 25 ms)
[ Info: Starting command: click_left (latency: 23 ms)
[ Info: Starting command: click_left (latency: 28 ms)
[ Info: Starting command: click_left (latency: 23 ms)
[ Info: Starting command: click_left (latency: 23 ms)
[ Info: Starting command: click_left (latency: 22 ms)
[ Info: Starting command: click_left (latency: 28 ms)
</details>

  • There are no community guidelines. I'd recommend linking something like https://github.com/SciML/ColPrac

  • Some comparisons to other implementations of a text-to-command software would be nice (in terms of API, performance, latency)

@carstenbauer
Copy link
Member

Hey @omlins, please take a look at @pfitzseb's suggestions and comments above and try to incorporate or comment on them.

@PetrKryslUCSD Any update on your review? Would be great to get this in soon. Thanks in advance.

@carstenbauer
Copy link
Member

@PetrKryslUCSD Can you please provide your review so that we can push this over the line?

@omlins Did you already incorporate the suggestions by @pfitzseb?

@omlins
Copy link

omlins commented Apr 21, 2023

@carstenbauer : I had addressed some of the issues only partially until recently as the second review is still pending. However, I have now been able to address all the issues raised by @pfitzseb conclusively, with exception of the last one.

Here, @pfitzseb and @carstenbauer, you find a reply to each issue raised:

* In Fig 2, a `using JustSayIt.API` is required (same goes for the documentation at https://omlins.github.io/JustSayIt.jl/stable/usage/

Some stale commands in the max_speed_subset had caused the error when executing the code in Fig 2 (resolved here: omlins/JustSayIt.jl#91). This error does not seem to be present in the documentation.

* [Usage example fails due to incorrect type checks omlins/JustSayIt.jl#87](https://github.com/omlins/JustSayIt.jl/issues/87) causes the example in Fig 2 to be non-functional.

This code issue has been solved and the corresponding pull request merged (omlins/JustSayIt.jl#89).

* [Keyboard.type fails with "insecure command argument recognition" omlins/JustSayIt.jl#88](https://github.com/omlins/JustSayIt.jl/issues/88) causes the `Keyboard.type` command to be non-functional.

This was not an issue with the code, but an incorrect usage of JustSayIt (omlins/JustSayIt.jl#88 (comment)).

* I'm seeing worse latency than shown in the article, but I'm happy to chalk that up to benchmarking on a signigicantly less powerful machine.

I believe it is right to expect that the latency will be different from system to system.

* There are no community guidelines. I'd recommend linking something like https://github.com/SciML/ColPrac

The contribution guide line has been extended and a link to the mentioned guideline added (omlins/JustSayIt.jl#92).

* Some comparisons to other implementations of a text-to-command software would be nice (in terms of API, performance, latency)

I would prefer to wait for the second review to address this issue, because additions to the text mean also removal of existing parts, given the very short format of this paper (we are already at the limit of the two pages).

In addition to these issues, I've also been able to solve the longstanding issue with the automatic installation not working in some corner cases, which @pfitzseb observed as well (resolved here: omlins/JustSayIt.jl#90). Note that independent of all the issues raised, two unit tests fail since recently on some versions of Ubuntu (the unit testing is complex, because the Vosk speech recognition tool kit used is not fully deterministic). They are not critical and could be removed in another pull request.

@matbesancon
Copy link
Member

ping @PetrKryslUCSD

@matbesancon
Copy link
Member

@whedon assign @matbesancon as editor

@lucaferranti
Copy link
Member

@editorialbot assign @matbesancon as editor

@editorialbot
Copy link
Collaborator

Assigned! @matbesancon is now the editor

@pfitzseb
Copy link
Collaborator

Alright, this ticks all the boxes now :)

@matbesancon
Copy link
Member

@editorialbot accept

@editorialbot
Copy link
Collaborator

Paper is not ready for acceptance yet, the archive is missing

@matbesancon
Copy link
Member

@omlins can you add the version and Zenodo DOI archive?

@omlins
Copy link

omlins commented Nov 5, 2024

@omlins can you add the version and Zenodo DOI archive?

I just saw the notification. Yes, I will do so.

@omlins
Copy link

omlins commented Nov 8, 2024

@matbesancon : here you can find the DOI for the paper: https://doi.org/10.5281/zenodo.14057465
here you can find the DOI for the code: https://doi.org/10.5281/zenodo.14057584

@matbesancon
Copy link
Member

@editorialbot accept

@editorialbot
Copy link
Collaborator 

Doing it live! Attempting automated processing of paper acceptance...

@editorialbot
Copy link
Collaborator

⚠️ Couldn't acccept/publish paper. An error happened. Paper file not found.

@matbesancon
Copy link
Member

@editorialbot recommend-accept

@editorialbot
Copy link
Collaborator 

Attempting dry run of processing paper acceptance...

@editorialbot
Copy link
Collaborator

Checking the BibTeX entries failed with the following error:

No paper file path

@editorialbot
Copy link
Collaborator

⚠️ Error preparing paper acceptance. Paper file not found.

@matbesancon
Copy link
Member

@editorialbot generate pdf from branch JuliaConProceeding2022

@editorialbot
Copy link
Collaborator

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@editorialbot commands

@matbesancon
Copy link
Member

@editorialbot generate pdf

@editorialbot
Copy link
Collaborator

⚠️ An error happened when generating the pdf. Paper file not found.

@matbesancon
Copy link
Member

@editorialbot generate pdf

@editorialbot
Copy link
Collaborator

⚠️ An error happened when generating the pdf. Paper file not found.

@matbesancon
Copy link
Member

@editorialbot generate pdf from branch JuliaConProceeding2022

@editorialbot
Copy link
Collaborator

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@editorialbot commands

@matbesancon
Copy link
Member

@editorialbot generate pdf

@editorialbot
Copy link
Collaborator

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

@matbesancon
Copy link
Member

@editorialbot set branch JuliaConProceeding2022

@editorialbot
Copy link
Collaborator

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@editorialbot commands

@matbesancon
Copy link
Member

@editorialbot recommend-accept

@editorialbot
Copy link
Collaborator 

Attempting dry run of processing paper acceptance...

@editorialbot
Copy link
Collaborator 

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- None

🟡 SKIP DOIs

- No DOI given, and none found for title: The Kaldi Speech Recognition Toolkit
- No DOI given, and none found for title: Vosk Speech Recognition Toolkit: Offline speech re...
- No DOI given, and none found for title: PyCall.jl: Package to call Python functions from t...
- No DOI given, and none found for title: Conda.jl: Conda managing Julia binary dependencies
- No DOI given, and none found for title: MacroTools.jl: a library of tools for working with...
- No DOI given, and none found for title: sounddevice: Play and Record Sound with Python
- No DOI given, and none found for title: pynput

❌ MISSING DOIs

- None

❌ INVALID DOIs

- None

@matbesancon
Copy link
Member

@editorialbot accept

@editorialbot
Copy link
Collaborator 

Doing it live! Attempting automated processing of paper acceptance...

@editorialbot
Copy link
Collaborator

👋 @JuliaCon/jcon-eics, this paper is ready to be accepted and published.

Check final proof 👉📄 Download article

If the paper PDF and the deposit XML files look good in JuliaCon/proceedings-papers#115, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

@matbesancon
Copy link
Member

@omlins please check the paper before the final acceptance!

@editorialbot
Copy link
Collaborator

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JCON! 🚨🚨🚨

Here's what you must now do:

  1. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.jcon.00121 proceedings-papers#116
  2. Wait five minutes, then verify that the paper DOI resolves https://doi.org/10.21105/jcon.00121
  3. If everything looks good, then close this review issue.
  4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@matbesancon matbesancon

Labels
accepted Julia published recommend-accept review
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@carstenbauer @pfitzseb @matbesancon @whedon @omlins @lucaferranti @editorialbot