Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update searches.py to refresh when blocked #87

Merged
merged 2 commits into from
Mar 14, 2024

Conversation

Tiwing
Copy link

@Tiwing Tiwing commented Mar 14, 2024

hello, I use this function when searches take too long to refresh the page, and unlock the timer.

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @Tiwing - I've reviewed your changes and they look great!

General suggestions:

  • Consider implementing a maximum attempt limit or a more sophisticated back-off strategy to prevent potential infinite loops.
  • Verify the assumption that points should always increase after each search to avoid unnecessary page refreshes.
  • Enhance logging messages with more context to aid in troubleshooting.
  • Evaluate the implications of refreshing the page on the web driver's state and explore alternative strategies for resolving blockages.
  • Encapsulate the retry logic into a separate method or use a generic retry mechanism to improve code readability and reusability.
Here's what I looked at during the review
  • 🟡 General issues: 5 issues found
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟢 Complexity: all looks good
  • 🟢 Docstrings: all looks good

Thanks for using Sourcery. We offer it for free for open source projects and would be very grateful if you could help us grow. If you like it, would you consider sharing Sourcery on your favourite social media? ✨

Share Sourcery

Help me be more useful! Please click 👍 or 👎 on each comment to tell me if it was helpful.

Comment on lines 77 to +88
pointsCounter = points
else:
break

if points <= pointsCounter:
attempt += 1
if attempt == 2:
logging.warning(
"[BING] Possible blockage. Refreshing the page."
)
self.webdriver.refresh()
attempt = 0
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code_refinement): Introducing a retry mechanism based on the number of attempts is a good resilience strategy. However, resetting the attempt counter to 0 after refreshing the page might not always be the best approach. Consider scenarios where the page refresh does not resolve the issue, leading to a potential infinite loop if the condition triggering the refresh persists. It might be beneficial to implement a maximum attempt limit or a more sophisticated back-off strategy.

@@ -76,6 +77,15 @@
pointsCounter = points
else:
break

if points <= pointsCounter:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question (code_clarification): The condition if points <= pointsCounter for triggering a page refresh seems to rely on the assumption that points should always increase after each search. While this might be true in most cases, it's important to verify this assumption against the application's behavior or external dependencies like the Bing search engine. If there are legitimate scenarios where points might not increase (e.g., rate limiting, daily caps), this could lead to unnecessary page refreshes.

if points <= pointsCounter:
attempt += 1
if attempt == 2:
logging.warning(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code_refinement): Logging a warning when a possible blockage is detected is a good practice for monitoring and debugging. However, consider enhancing the log message with more context, such as the current value of points and pointsCounter, or even the search term that led to this situation. This additional information can be invaluable during troubleshooting.

logging.warning(
"[BING] Possible blockage. Refreshing the page."
)
self.webdriver.refresh()
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code_refinement): Using self.webdriver.refresh() to attempt to resolve a blockage is a straightforward approach. However, consider the implications of this action on the state of the web driver and the current session. For instance, refreshing might reset any transient state or cookies that are essential for the search operation. It might be worth exploring alternative strategies, such as navigating to a different page or restarting the search session entirely.

"[BING] Possible blockage. Refreshing the page."
)
self.webdriver.refresh()
attempt = 0
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code_refinement): Resetting the attempt counter to 0 after a page refresh is a clear and understandable action. However, this logic is tightly coupled with the loop's structure and the specific condition being checked. Consider encapsulating this retry logic into a separate method or using a more generic retry mechanism. This could improve code readability and make the retry logic reusable across different parts of the application.

@klept0 klept0 merged commit efb7537 into klept0:master Mar 14, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants