Support distant:// scheme?

[chipsenkbeil]

Any thoughts on supporting distant (https://distant.rocks) as another option for remote editing? It offers a JSON lines API that you can used for authentication, file I/O, process management (including running LSP servers), remote shells, file/directory watching, and automatic reconnecting for poor networking.

I've built out distant.nvim to leverage that CLI to both support SSH and the custom distant protocol, but would also love to see more adoption across fellow plugins and integrations to enable remote editing features.

[miversen33]

Ironically I looked at your plugin before creating netman!

Because of how netman is structured, a distant provider can absolutely be created for netman. My provider creation docs aren't great though. If you're willing to provide some assistance/long term maintenance with the provider, I have no issue in including that provider in core :)

I will not include it in the 1.15 or 1.2 milestones, imo something like this would be it's own work effort altogether. But I am very much behind adding this kind of support!

[chipsenkbeil]

Alright, so do you recommend revisiting this later? 😄

[miversen33]

It depends on what you mean by "revisit". I am not (at this time) looking to create this provider and I don't know if that changes in the future. I am very much fine with supporting someone else's contribution of said provider (including helping them navigate the API, any confusion they have, etc). The latter can be done at anytime, in which case I don't know that revisiting makes sense unless the contributor wants to do so later :)

The long short, I think your idea is good. I don't think I have the energy or time to do it (and I don't know enough about distant in order to feel comfortable maintaining it).
I do know enough about Netman to provide whatever support is required for someone else to create and maintain that provider, and adding it to to core is acceptable (as I imagine this would be usable by many) though technically this can its own standalone plugin (a process that is documented in the developer doc though may be a bit confusing)

[chipsenkbeil]

Ah, okay. I misunderstood your earlier comments as needing to wait for documentation and reaching a later milestone. I'll take a look at the plugin docs, but my interest would be to avoid maintaining a second plugin myself (e.g. the standalone plugin route), so if I can help integrate the feature into netman directly and provide answers/support in this repo, that'd be my preference.

How would you recommend proceeding when it comes to integrating into core? Any code pointers for me to look at?

[miversen33]

I'll take a look at the plugin docs, but my interest would be to avoid maintaining a second plugin myself (e.g. the standalone plugin route), so if I can help integrate the feature into netman directly and provide answers/support in this repo, that'd be my preference.

Its worth mentioning that distant.nvim could actually just integrate itself into netman to avoid having a separate plugin :)

Anyway! To your question
The best way to get started is the developer doc I linked above, it walks through how to create an (admittedly out of date) docker provider.
Alternatively, you can checkout the docker provider in core.

Adding a new provider to core should be reasonably simple.

• Create a new provider in netman.providers (we can call it distant.lua)
• Add the path to this new provider to netman.providers.init
• Implement the required attributes and functions for a provider. Details on this can be found in the api doc.

The above should get you an end result of a provider that is available by default in Netman, which is what we want with core providers.

I would say the best "code" starting point is the docker provider which is what I would call the "most complete" provider.

Please note, per #117 , Netman is purely synchronous currently. There will be optional functions that a provider can implement if it wishes to support asynchronous behavior, but I do not have them fleshed out yet.

Feel free to reach out to me on discord miversen33#0131 if you have questions and would like "more immediate" assistance on how to do things :) I know parsing doc and someone else's code can be... Painful at times.

And thank you! I have been hoping others would have interest in creating providers for netman, (as well as consumers but I haven't fleshed that out just yet). Please let me know how I can help you

Edit:
I forgot to mention, there is a lua bootstrapper I created (located at netman.tools.bootstrap) that might allow you to test your provider without having to spin up a new instance of neovim each time. You can (if you load the bootstrapper first) actually run your provider directly in a lua shell. This will only work if you use the netman.tools.shell (instead of plenary.jobs or vim anything).

Which is probably important to mention.
This project doesn't have any external dependencies and that is part of its goal. There is the obvious dependency for this provider to function (distant must be available in order to use it) and that is acceptable (and assumed). But to be part of core, a provider must not require anything outside of what vim and netman provide in order to operate. In short, having a requirement of distant is fine as this would be a provider to talk to distant. If distant doesn't exist then of course the provider doesn't work. However requiring plenary.nvim (as an example) is not acceptable as that is not required to talk to distant. If that makes sense

3rd party plugins (what distant.nvim would be considered if it added netman integration) can absolutely use external dependencies (such as plenary). But I want netman itself to function without those kinds of requirements. Something to consider if you want this integration to be part of core.

[chipsenkbeil]

I can appreciate the strictness on dependencies as distant.nvim also avoids needing any additional dependencies, although I do make concessions by providing integrations into other plugins like telescope.nvim, but they're all optional.

So it sounds like if I want to have distant be available as a provider in core, I would need to write the API communication again rather than importing distant.nvim, which already supports synchronous and asynchronous APIs. That's not a dealbreaker for me, but want to make sure I understood your explanation before I commit to one approach versus the other!

I was skimming through your documentation and the docker provider, and didn't immediately see a standard way to support connection setup and/or authentication. For the distant plugin, when you launch a new server or connect to an existing server, a handshake is performed that includes zero or more authentication challenges. My first guess is that I could provide this synchronously through the optional init function?

[miversen33]

So it sounds like if I want to have distant be available as a provider in core, I would need to write the API communication again rather than importing distant.nvim, which already supports synchronous and asynchronous APIs. That's not a dealbreaker for me, but want to make sure I understood your explanation before I commit to one approach versus the other!

This is correct unfortunately. It may be easier to add a bit of wrapper code to distant.nvim to integrate with netman (thus making netman an optional dependency of distant.nvim, like you do for telescope for example), in which case you don't have to completely port your code over to Netman. It may not be easier though, I don't know lol. I am happy to provide support in either decision though :)

I was skimming through your documentation and the docker provider, and didn't immediately see a standard way to support connection setup and/or authentication. For the distant plugin, when you launch a new server or connect to an existing server, a handshake is performed that includes zero or more authentication challenges. My first guess is that I could provide this synchronously through the optional init function?

Ya I haven't landed on exactly how I want connections to work unfortunately. I am currently toying with this as the core ssh plugin could benefit from a "connect" function as well. For the time being, I execute the connect logic automagically with SSH_MULTIPLEXING (meaning SSH handles the initial connection and I am able to just assume the pipe exists). However there are definitely times that that hurts.

The optional "init" function is called on the initial load of the plugin (as seen in api.load_provider). This is called (most likely) during neovim startup and gives your provider an opportunity to verify it can run (all dependencies are met for example). It is not called on initial read of a URI as the API has no concept of URI connection states.

Currently (until/unless an optional connect method is introduced to the provider spec), your best bet would be to have an object that manages your distant information (including if its connected) and perform the connection whenever it recognizes it isn't connected. I don't know what that logic looks like for distant, but that is how I would go about handling that within the Netman environment.

I want to call out, I am not against a adding this kind of functionality (so if you have an idea of what you would like it to look like, I would be fine with chatting about that spec), however as I call out in the api documentation the API should be as stateless as possible and as dumb as possible. Thus implementing some kind of URI connection logic would likely need to be very well thought through as I am not open to having the API manage if a URI is connected or not (as that is what the provider should do).

I hope that answered your questions :)

[chipsenkbeil]

Yeah, thanks for the detailed responses here :) Figure having questions answered in this issue (versus discord chat) would be better for others who inquire later.

For distant, I actually mirrored ssh's method of authentication (being in protocol and client APIs), so I'd be happy to chat about how that works in the future. Lazy validation of authentication seems like a path forward to me, so I can certainly implement that approach for netman.

[miversen33]

Makes sense to me! Let me know if/when you need anything. I would fork off of main as opposed to v1.15 (if you are unsure which branch to use). Anything not main is unstable and subject to being yeeted off at anytime. I think I am going to close this issue out as it sounds like we have a general plan on what to do for this. Feel free to open new issues about whatever obscure (or not) bugs you encounter :)