Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.2.0 #208

Draft
wants to merge 93 commits into
base: master
Choose a base branch
from
Draft

1.2.0 #208

Show file tree
Hide file tree
Changes from 47 commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
05611c7
Update morePromotions.py
OmegaBlurz Sep 3, 2024
4400d48
Update dailySet.py
Rippenkneifer Sep 6, 2024
2b4ea72
Update activities.py
Rippenkneifer Sep 6, 2024
cd30d85
Update login.py
Rippenkneifer Sep 6, 2024
c793db2
Pulled Pork - Bann & Lock check (#201)
cal4 Sep 7, 2024
39bdfea
Pork shoulders (#200)
cal4 Sep 7, 2024
568a721
Refactor getRemainingSearches
cal4 Sep 7, 2024
cb3c668
Add fixme
cal4 Sep 7, 2024
4560eae
Put logging back
cal4 Sep 7, 2024
2986396
.gitignore
Guido30 Sep 7, 2024
f8f29b3
removed config.yaml
Guido30 Sep 7, 2024
102d90b
Merged config-private into config.yaml
Guido30 Sep 7, 2024
dcdfdce
Restored configurarion files
Guido30 Sep 8, 2024
56c6721
More configs for apprise and more (#203)
cal4 Sep 8, 2024
cc7d4c2
Update morePromotions.py
OmegaBlurz Sep 9, 2024
4d8ed73
First Docker implementation
belgio99 Sep 9, 2024
ab079fc
Removed duplicate options from cherry-pick
belgio99 Sep 9, 2024
245adb1
Added first GitHub Action to push on Docker Hub
belgio99 Sep 9, 2024
b2160cd
Added Docker image versioning based on GitHub tags
belgio99 Sep 9, 2024
97a4616
Added 15s timeout to support 2FA
belgio99 Sep 9, 2024
7893e12
chore: Update Dockerfile to use apt-get instead of apt, and pip --no-…
belgio99 Sep 9, 2024
653392f
Delete the apt-get lists after installing packages
belgio99 Sep 9, 2024
b08ae39
Removed unused 2FA import
belgio99 Sep 9, 2024
c8082ef
Fixed Docker semantic versioning
belgio99 Sep 9, 2024
57560e8
Add more promos (#205)
klept0 Sep 9, 2024
ab4a8a7
Change action to match username specified in secret
belgio99 Sep 9, 2024
fbccd4d
Docker implementation (#114)
cal4 Sep 9, 2024
fa5194f
Update morePromotions.py
klept0 Sep 9, 2024
ed8c7d6
Add config singletons, combine default configs
cal4 Sep 9, 2024
f726e57
Make search simpler and remove failing terms completely
cal4 Sep 9, 2024
f8d5ba9
Change log level to info
cal4 Sep 9, 2024
8558132
Add assertion message
cal4 Sep 9, 2024
04e9e36
Make quicker
cal4 Sep 9, 2024
8a9df3d
Fix indent
cal4 Sep 9, 2024
1e90f13
Add new promo and fix some
cal4 Sep 10, 2024
de76983
Reformat and add PROMOTION_TITLE_TO_SEARCH
cal4 Sep 10, 2024
3798ecb
Add comment
cal4 Sep 11, 2024
14cc1d7
Add jitter and remove delete
cal4 Sep 11, 2024
93375b1
Fix error when switching to new tab and optionally close it
cal4 Sep 11, 2024
885f19c
Clarify lang and geo options
cal4 Sep 12, 2024
6b3c39a
Reformat
cal4 Sep 12, 2024
d3bbaa6
Get language and country from locale
cal4 Sep 12, 2024
ee86e6d
Add pycountry and sort alphabetically
cal4 Sep 12, 2024
ac792f5
Remove ipapi and upgrade apprise
cal4 Sep 12, 2024
f79be71
Refactor getting language and country
cal4 Sep 14, 2024
f6a9feb
Refactor warning logging
cal4 Sep 14, 2024
49e4930
Put ipapi back
cal4 Sep 14, 2024
7f6080a
Handle all activities in single loop
cal4 Sep 27, 2024
9a3dc6c
Refactor config
cal4 Sep 27, 2024
bdd0c86
Clean up and make consistent
cal4 Sep 27, 2024
38f6c12
Simplify message dismiss
cal4 Sep 27, 2024
fed3068
Simplify message dismiss
cal4 Sep 27, 2024
478557d
Simplify message dismiss
cal4 Sep 27, 2024
0e2c966
Use typing_extensions
cal4 Sep 27, 2024
f89f2d2
Add option to ignore identity activity
cal4 Sep 27, 2024
494c9df
Reformat using Black
cal4 Sep 27, 2024
44b0116
Alphabetize
cal4 Oct 23, 2024
bc30d82
Clean up activity title for emails
cal4 Oct 23, 2024
92d2720
Fix issue where page load timed out
cal4 Oct 23, 2024
4572c60
Refactor imports and use method name only
cal4 Oct 23, 2024
9291a6e
Remove staticmethod in favor of function https://stackoverflow.com/a/…
cal4 Oct 23, 2024
ff3b02d
Remove staticmethod in favor of function https://stackoverflow.com/a/…
cal4 Oct 23, 2024
a15f2e5
Add fixme
cal4 Nov 2, 2024
c936439
Remove deprecated classes
cal4 Nov 5, 2024
6bed6fc
Up timeToWait and use default
cal4 Nov 5, 2024
50dc6dc
Fix import order
cal4 Nov 6, 2024
8f6ec04
Increase base_delay_in_seconds
cal4 Nov 7, 2024
d9d6e7d
Sleep 5-10 minutes after success
cal4 Nov 14, 2024
68602ed
Get points from dashboard since more reliable
cal4 Nov 14, 2024
ef098d9
Remove scrolls
cal4 Nov 14, 2024
245d3b1
Remove comment
cal4 Nov 15, 2024
ef42d25
Log incomplete activities
cal4 Nov 17, 2024
81afb32
Check for level before getting remaining mobile searches
cal4 Nov 17, 2024
6511198
Fix Discover open job roles
cal4 Nov 18, 2024
2e2540c
Add new activities
cal4 Nov 25, 2024
815001a
Add apprise notification on login with phone code
Kyrela Nov 29, 2024
b6ee600
Add login notifications configuration in config.yaml
Kyrela Dec 1, 2024
c269afb
Add apprise notification on login with phone code (#229)
cal4 Dec 1, 2024
0785ba2
Update userAgentGenerator.py
jdeath Dec 3, 2024
68e4c7f
Update userAgentGenerator.py
jdeath Dec 3, 2024
a1352d8
Update userAgentGenerator.py
jdeath Dec 3, 2024
9a21404
Fix Edge API (#233)
cal4 Dec 3, 2024
7974152
fix: resolve KeyError in getEdgeVersions and bug in getLanguageCountry
SteveTryndamere Dec 8, 2024
6d23c75
Handle when daily_set_date exists but is empty
cal4 Dec 9, 2024
2bf51d3
Add ability to more generically ignore activities
cal4 Dec 9, 2024
4ff9d79
fix: resolve KeyError in getEdgeVersions and bug in getLanguageCountr…
cal4 Dec 10, 2024
c6d5889
Exit 1 on exception
Kyrela Dec 14, 2024
63c6b32
Better, simpler and unified config
Kyrela Dec 17, 2024
a3d2aac
Enhance README.md with command-line override details
Kyrela Dec 25, 2024
8604039
Exit 1 on error (#244)
klept0 Jan 6, 2025
770e318
[PROPOSAL] Better, simpler and unified config (#248)
klept0 Jan 21, 2025
e88f6ee
Fix unwanted changed on #237
Kyrela Jan 22, 2025
1671870
Fix unwanted changes on #237 (#258)
klept0 Jan 25, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/build_push_docker_image.yml

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this "docker" functionality affect detection at all? I know that running chromium headless can increase chances of being caught by bing, has this been changed?

Btw for alternatives to docker for this usecase, i've already investigate nix, but it does not have the packages required and selenium-wire is broken iirc.

Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Build and Push Docker Image

on:
push:
tags:
- "*"
workflow_dispatch:

jobs:
build:
runs-on: ubuntu-latest

steps:
- name: Checkout Code
uses: actions/checkout@v4
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ secrets.DOCKERHUB_USERNAME }}/ms-rewards
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Login to DockerHub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
9 changes: 4 additions & 5 deletions .template-config-private.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
# config-private.yaml
# Copy this file to config-private.yaml to use
apprise:
urls:
- 'discord://WebhookID/WebhookToken' # Replace with your actual Apprise service URLs
# config-private.yaml
apprise:
urls:
- "discord://{WebhookID}/{WebhookToken}" # Replace with your actual Apprise service URLs
9 changes: 9 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
FROM python:slim
COPY . /app
WORKDIR /app
RUN apt-get update && apt-get install -y cron chromium chromium-driver \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
RUN pip install --no-cache-dir -r requirements.txt
ENV DOCKER=1
CMD ["sh", "docker.sh"]
52 changes: 31 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@


> [!IMPORTANT]
> If you are multi-accounting and abusing the service for which this is intended - **_DO NOT COMPLAIN ABOUT BANS!!!_**
> If you are multi-accounting and abusing the service for which this is intended - *
*_DO NOT COMPLAIN ABOUT BANS!!!_**



Expand Down Expand Up @@ -42,18 +43,21 @@
3. (Windows Only) Make sure Visual C++ redistributable DLLs are installed

If they're not, install the current "vc_redist.exe" from
this [link](https://learn.microsoft.com/en-GB/cpp/windows/latest-supported-vc-redist?view=msvc-170) and reboot your
computer
this [link](https://learn.microsoft.com/en-GB/cpp/windows/latest-supported-vc-redist?view=msvc-170)
and reboot your computer

4. Edit the `accounts.json.sample` with your accounts credentials and rename it by removing `.sample` at the end.
4. Edit the `.template-config-private.yaml` accordingly and rename it to `config-private.yaml`.

The "totp" field is not mandatory, only enter your TOTP key if you use it for 2FA (if ommitting, don't keep
it as an empty string, remove the line completely).
5. Edit the `accounts.json.sample` with your accounts credentials and rename it by removing
`.sample` at the end.

The "proxy" field is not mandatory, you can omit it if you don't want to use proxy (don't keep it as an empty string,
remove the line completely).
The "totp" field is not mandatory, only enter your TOTP key if you use it for 2FA (if
ommitting, don't keep it as an empty string, remove the line completely).

- If you want to add more than one account, the syntax is the following:
The "proxy" field is not mandatory, you can omit it if you don't want to use proxy (don't
keep it as an empty string, remove the line completely).

- If you want to add more than one account, the syntax is the following:

```json
[
Expand All @@ -72,29 +76,34 @@
]
```

5. Run the script:
6. Run the script:

`python main.py`

6. (Windows Only) You can set up automatic execution by generating a Task Scheduler XML file.

If you are a Windows user, run the `generate_task_xml.py` script to create a `.xml` file. After generating the file, import it into Task Scheduler to schedule automatic execution of the script. This will allow the script to run at the specified time without manual intervention.
7. (Windows Only) You can set up automatic execution by generating a Task Scheduler XML file.

To import the XML file into Task Scheduler, see [this guide](https://superuser.com/a/485565/709704).
If you are a Windows user, run the `generate_task_xml.py` script to create a `.xml` file.
After generating the file, import it into Task Scheduler to schedule automatic execution of
the script. This will allow the script to run at the specified time without manual
intervention.

To import the XML file into Task Scheduler,
see [this guide](https://superuser.com/a/485565/709704).

## Launch arguments

- `-v/--visible` to disable headless
- `-l/--lang` to force a language (ex: en)
- `-l/--lang` to force a language (ex: en) see https://serpapi.com/google-languages for options
- `-g/--geo` to force a searching geolocation (ex: US)
see https://serpapi.com/google-trends-locations for options
`https://trends.google.com/trends/ for proper geolocation abbreviation for your choice. These MUST be uppercase!!!`
- `-p/--proxy` to add a proxy to the whole program, supports http/https/socks4/socks5 (overrides per-account proxy in
accounts.json)
- `-p/--proxy` to add a proxy to the whole program, supports http/https/socks4/socks5 (
overrides per-account proxy in accounts.json)
`(ex: http://user:pass@host:port)`
- `-cv/--chromeversion` to use a specific version of chrome
`(ex: 118)`
- `-da/--disable-apprise` disables Apprise notifications for the session, overriding [config.yaml](config.yaml).
- `-da/--disable-apprise` disables Apprise notifications for the session,
overriding [config.yaml](config.yaml).
Useful when running manually as opposed to on a schedule.
- `-t/--searchtype` to only do `desktop` or `mobile` searches, `(ex: --searchtype=mobile)`

Expand All @@ -108,7 +117,8 @@
- Multi-Account Management
- Session storing
- 2FA Support
- Notifications via [Apprise](https://github.com/caronc/apprise) - no longer limited to Telegram or Discord
- Notifications via [Apprise](https://github.com/caronc/apprise) - no longer limited to
Telegram or Discord
- Proxy Support (3.0) - they need to be **high quality** proxies
- Logs to CSV file for point tracking

Expand All @@ -117,8 +127,8 @@
Fork this repo and:

* if providing a bugfix, create a pull request into master.
* if providing a new feature, please create a pull request into develop. Extra points if you update
the [CHANGELOG.md](CHANGELOG.md).
* if providing a new feature, please create a pull request into develop. Extra points if you
update the [CHANGELOG.md](CHANGELOG.md).

## To Do List (When time permits or someone makes a PR)

Expand Down
9 changes: 8 additions & 1 deletion config.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
# config.yaml
apprise:
notify:
incomplete-promotions: True # True or False
uncaught-exceptions: True # True or False
summary: ON_ERROR
default:
geolocation: US # Replace with your country code https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2
logging:
level: INFO # See https://docs.python.org/3/library/logging.html#logging-levels
retries:
base_delay_in_seconds: 14.0625 # base_delay_in_seconds * 2^max = 14.0625 * 2^6 = 900 = 15 minutes
max: 8
max: 4
strategy: EXPONENTIAL
32 changes: 32 additions & 0 deletions docker.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/bin/bash

# Check if RUN_ONCE environment variable is set. In case, running the script now and exiting.
if [ "$RUN_ONCE" = "true" ]
then
echo "RUN_ONCE environment variable is set. Running the script now and exiting."
python main.py
exit 0
fi
# Check if CRON_SCHEDULE environment variable is set
if [ -z "$CRON_SCHEDULE" ]
then
echo "CRON_SCHEDULE environment variable is not set. Setting it to 4 AM everyday by default"
CRON_SCHEDULE="0 4 * * *"
fi

# Setting up cron job
echo "$CRON_SCHEDULE root python /app/main.py >> /var/log/cron.log 2>&1" > /etc/cron.d/rewards-cron-job

# Give execution rights on the cron job
chmod 0644 /etc/cron.d/rewards-cron-job

# Apply cron job
crontab /etc/cron.d/rewards-cron-job

# Create the log file to be able to run tail
touch /var/log/cron.log

echo "Cron job is set to run at $CRON_SCHEDULE. Waiting for the cron to run..."

# Run the cron
cron && tail -f /var/log/cron.log
15 changes: 6 additions & 9 deletions main.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
)
from src.browser import RemainingSearches
from src.loggingColoredFormatter import ColoredFormatter
from src.utils import Utils
from src.utils import Utils, CONFIG


def main():
Expand All @@ -40,10 +40,8 @@ def main():
earned_points = executeBot(currentAccount, args)
except Exception as e1:
logging.error("", exc_info=True)
Utils.sendNotification(
f"⚠️ Error executing {currentAccount.username}, please check the log",
traceback.format_exc(),
)
Utils.sendNotification(f"⚠️ Error executing {currentAccount.username}, please check the log",
traceback.format_exc(), e1)
continue
previous_points = previous_points_data.get(currentAccount.username, 0)

Expand Down Expand Up @@ -98,15 +96,14 @@ def setupLogging():
logs_directory.mkdir(parents=True, exist_ok=True)

# so only our code is logged if level=logging.DEBUG or finer
# if not working see https://stackoverflow.com/a/48891485/4164390
logging.config.dictConfig(
{
"version": 1,
"disable_existing_loggers": True,
}
)
logging.basicConfig(
level=logging.DEBUG,
level=logging.getLevelName(CONFIG.get("logging").get("level").upper()),
format=_format,
handlers=[
handlers.TimedRotatingFileHandler(
Expand Down Expand Up @@ -281,7 +278,7 @@ def executeBot(currentAccount: Account, args: argparse.Namespace):
f"[POINTS] You are now at {Utils.formatNumber(accountPoints)} points !"
)
appriseSummary = AppriseSummary[
Utils.loadConfig().get("apprise", {}).get("summary", AppriseSummary.ALWAYS.name)
CONFIG.get("apprise").get("summary")
]
if appriseSummary == AppriseSummary.ALWAYS:
goalStatus = ""
Expand Down Expand Up @@ -357,5 +354,5 @@ def save_previous_points_data(data):
except Exception as e:
logging.exception("")
Utils.sendNotification(
"⚠️ Error occurred, please check the log", traceback.format_exc()
"⚠️ Error occurred, please check the log", traceback.format_exc(), e
)
23 changes: 12 additions & 11 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
requests~=2.32.3
selenium>=4.15.2 # not directly required, pinned by Snyk to avoid a vulnerability
ipapi~=1.0.4
undetected-chromedriver==3.5.5
selenium-wire~=5.1.0
apprise~=1.9.0
blinker==1.7.0 # prevents issues on newer versions
numpy>=1.22.2 # not directly required, pinned by Snyk to avoid a vulnerability
setuptools
ipapi~=1.0.4
psutil
blinker==1.7.0 # prevents issues on newer versions
apprise~=1.8.1
pycountry~=24.6.1
pyotp~=2.9.0
pyyaml~=6.0.2
urllib3>=2.2.2 # not directly required, pinned by Snyk to avoid a vulnerability
requests-oauthlib~=2.0.0
zipp>=3.19.1 # not directly required, pinned by Snyk to avoid a vulnerability
pyotp~=2.9.0
requests~=2.32.3
selenium-wire~=5.1.0
selenium>=4.15.2 # not directly required, pinned by Snyk to avoid a vulnerability
setuptools
undetected-chromedriver==3.5.5
urllib3>=2.2.2 # not directly required, pinned by Snyk to avoid a vulnerability
zipp>=3.19.1 # not directly required, pinned by Snyk to avoid a vulnerability
28 changes: 27 additions & 1 deletion src/activities.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
import contextlib
import logging
import random
import time

from selenium.common import TimeoutException
from selenium.common.exceptions import NoSuchElementException, ElementNotInteractableException
from selenium.webdriver.common.by import By
from selenium.webdriver.remote.webelement import WebElement

Expand All @@ -14,6 +16,29 @@ def __init__(self, browser: Browser):
self.browser = browser
self.webdriver = browser.webdriver

def click_element_if_visible(self, element):
try:
if element.is_displayed() and element.is_enabled():
element.click()
logging.info("Dashboard pop-up registered and closed, needs to be done once on new accounts")
else:
pass
except (ElementNotInteractableException, NoSuchElementException):
pass

def dashboardPopUpModalCloseCross(self):
try:

element = self.webdriver.find_element(By.CSS_SELECTOR, ".dashboardPopUpPopUpSelectButton")
self.click_element_if_visible(element)
time.sleep(0.25)
except NoSuchElementException:
return



Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: formatting. run ruff on this :)



def openDailySetActivity(self, cardId: int):
# Open the Daily Set activity for the given cardId
element = self.webdriver.find_element(By.XPATH,
Expand All @@ -26,7 +51,7 @@ def openMorePromotionsActivity(self, cardId: int):
element = self.webdriver.find_element(By.CSS_SELECTOR,
f"#more-activities > .m-card-group > .ng-scope:nth-child({cardId + 1}) .ds-card-sec")
self.browser.utils.click(element)
self.browser.utils.switchToNewTab(timeToWait=5)
self.browser.utils.switchToNewTab(timeToWait=8)

def completeSearch(self):
# Simulate completing a search activity
Expand All @@ -45,6 +70,7 @@ def completeQuiz(self):
with contextlib.suppress(TimeoutException):
startQuiz = self.browser.utils.waitUntilQuizLoads()
self.browser.utils.click(startQuiz)
# this is bugged on Chrome for some reason
cal4 marked this conversation as resolved.
Show resolved Hide resolved
self.browser.utils.waitUntilVisible(
By.ID, "overlayPanel", 5
)
Expand Down
Loading