Feature Request: Implement Question-Based Crawler with Search Engine Integration

[TheCutestCat]

Hi,
I'd like to contribute to the Question-Based Crawler feature listed in the roadmap under "Natural language driven web discovery and content extraction". I propose implementing this using free search engine APIs (like DuckDuckGo) combined with LLM-based filtering of the retrieved results. The functionality could be implemented similarly to EXA's approach.

Questions/Discussion Points:

1. What would be the most appropriate location in the current codebase structure to implement these advanced features?
2. Which branch should this feature development be based on?

I would like to take on this task and contribute to its development. Looking forward to guidance on the architectural decisions before proceeding.