Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replay not using AUTHORIZATION TOKEN from token_refresh_command #848

Open
marcindulak opened this issue Jan 22, 2024 · 5 comments · May be fixed by #932
Open

Replay not using AUTHORIZATION TOKEN from token_refresh_command #848

marcindulak opened this issue Jan 22, 2024 · 5 comments · May be fixed by #932

Comments

@marcindulak
Copy link

Description

It appears that the replay of bugs doesn't use the AUTHORIZATION TOKEN, when the token is provided using --token_refresh_command and --token_refresh_interval parameters.

The token is used as expected, and confirmed to be valid, when the bug_buckets/*.replay.txt is modified by hand to replace AUTHORIZATION TOKEN by Authorization: Bearer RealTokenHere.

Steps to reproduce

These are not really reproduction steps, I hope someone will be able to confirm this using a project with Authorization: Bearer token.

I'm running a replay of a bug found by the fuzzer

/RESTler/restler/Restler replay --host app --target_port 8000 --grammar_file Compile/grammar.py \
   --dictionary_file Compile/dict.json --settings Compile/engine_settings.json --no_ssl \
   --token_refresh_command 'sh /usr/src/restler-fuzzer/authentication_token.sh' --token_refresh_interval 10800 \
   --replay_log Fuzz/RestlerResults/experiment525/bug_buckets/PayloadBodyChecker_500_1.replay.txt

The main contents of Fuzz/RestlerResults/experiment525/bug_buckets/PayloadBodyChecker_500_1.replay.txt is

-> PUT /api/v1/seed HTTP/1.1\r\nAccept: application/json\r\nHost: app\r\nContent-Type: application/json\r\nAUTHORIZATION TOKEN\r\n\r\nfalse
! producer_timing_delay 0
! max_async_wait_time 0
PREVIOUS RESPONSE: 'HTTP/1.1 500 Internal Server Error\r\ndate:  ...

The contents of /usr/src/restler-fuzzer/authentication_token.sh is

echo {'id':{}}
echo Authorization: Bearer RealTokenHere

If I replace AUTHORIZATION TOKEN with Authorization: Bearer RealTokenHere in the bug_buckets/*.replay.txt , then the bug replays as expected.

Expected results

Starting task Replay...
Using python: 'python3' (Python 3.11.6)
Task Replay succeeded.
Collecting logs...

Actual results

When using AUTHORIZATION TOKEN with --token_refresh_command and --token_refresh_interval, the fuzzer logs the following on screen

Starting task Replay...
Using python: 'python3' (Python 3.11.6)

ERROR: Restler engine failed. See logs in /usr/src/restler-fuzzer/Replay directory for more information. 

Task Replay failed.
Collecting logs...

and /usr/src/restler-fuzzer/Replay/EngineStdOut.txt contains

Failed to play sequence from log:
A valid authorization token was expected.
Retry with a token refresh script in the settings file or update the request in the replay log with a valid authorization token.

Environment details

PRETTY_NAME="Alpine Linux v3.18"; Python 3.11.6; No .NET SDKs were found; RESTler version: 9.2.3; aef01a5
@marcindulak
Copy link
Author

marcindulak commented Jan 22, 2024 •
edited
Loading

Tried to debug and in /RESTler/engine/engine/core/request_utilities.py added some debug prints

def replace_auth_token(data, replace_str): 
    print("replace_auth_token", data)                               
    print("replace_str", replace_str)                               
    if data:                                                
        if latest_token_value:                              
            print("latest_token_value", latest_token_value) 
            data = data.replace(latest_token_value.strip('\r\n'), replace_str)
        if latest_shadow_token_value:                                         
            print("latest_shadow_token_value", latest_shadow_token_value)     
            data = data.replace(latest_shadow_token_value.strip('\r\n'), replace_str)
    print("data", data)                                                              
    return data

This results in the following cat /usr/src/restler-fuzzer/Replay/EngineStdOut.txt

replace_auth_token Will refresh token: sh /usr/src/restler-fuzzer/authentication_token.sh
replace_str _OMITTED_AUTH_TOKEN_
latest_token_value NO-TOKEN-SPECIFIED

latest_shadow_token_value NO-SHADOW-TOKEN-SPECIFIED

Failed to play sequence from log:
A valid authorization token was expected.
Retry with a token refresh script in the settings file or update the request in the replay log with a valid authorization token.

@marcindulak
Copy link
Author

@marina-p

@DarkTinia
Copy link

I also encountered the same problem.
When I set cmd it fails to replay but when I change AUTHORIZATION TOKEN it replays fine.

@Lamuji
Copy link

Lamuji commented Dec 4, 2024

Hi,

Same here. Did you find where the pb is ?

@dmaciejak
Copy link

dmaciejak commented Dec 18, 2024 •
edited
Loading

Replay is also supposed to support --settings but if you are passing authentication token via the engine_settings.json file it does not work either on my side

vai-alkos added a commit to vai-alkos/restler-fuzzer that referenced this issue Jan 15, 2025
@vai-alkos
Fix microsoft#848: Use authorization token from token_refresh_command…
3c05d43 
… during replay

Replace execute_token_refresh_cmd with execute_token_refresh
to ensure the token is refreshed and set as latest_token_value.
vai-alkos added a commit to vai-alkos/restler-fuzzer that referenced this issue Jan 15, 2025
@vai-alkos
Fix microsoft#848: Use auth token from token_refresh_command during r…
f995e93 
…eplay

Replace execute_token_refresh_cmd with execute_token_refresh
to ensure the token is refreshed and set as latest_token_value.
@vai-alkos vai-alkos linked a pull request Jan 15, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use authorization token from token_refresh_command during replay vai-alkos/restler-fuzzer
4 participants
@dmaciejak @marcindulak @DarkTinia @Lamuji