xml parsing error #53
Comments
|
gogrepoc worked for me 5 days ago. However, today I get the same "xml parsing error occurred trying to get md5 data" error. |
|
I'm not hugely familiar with Python but having experienced the same issue I threw a few The problem seems to be in function This is an example of a URL I intercepted (with the token blatted)
There's no query string here, so the MD5 function actually just downloads the full executable and then tries to parse it as XML which clearly isn't going to fly. A really dirty hack given I don't know any string handling functions in Python was to comment out that line and replace it with this one This assumes that there is no query and basically just slaps the .XML onto the end. As I write this response the downloader is whirring away quite successfully. Obviously this needs fixing properly to do a check to see if a query string is present or not before deciding how to process the URL, hence no PR - even though this is working for me I don't even know if this is the way you're supposed to concat strings in Python 🤷 . Still... hope it helps. |
|
It is possible to add the file extensions to SKIP_MD5_FILE_EXT in line 200 as a workaround. That skips MD5 checking completely and the "update" and "download" commands will work again. |
|
@z4zen Yeah, that works. I gave it a try since I was getting the XML parsing error while trying to update Cyberpunk 2077, and now I can update the manifest and download the game without any issues. |
Interesting, as I'm having the same issues with lgogdownloader as of today being unable to parse any XML file. |
|
Thank you @cyotek for pointing out the problematic place! I've created pull requests with something that looks to me like a little bit more robust fix 🙂 |
|
https://github.com/Sude-/lgogdownloader/issues/214 <-- lgogdownloader fixed it… |
|
@Tokariew thanks for a link As far as I know the requests library used in this script should handle gzip and deflate encoding of responses (here is related doc https://2.python-requests.org/en/master/user/quickstart/#response-content). So I don't think there is need to implement anything similar and decoding itself should already work out of the box. I have encountered problems with xml parsing for some games after PR (one of them is superhot_mind_control_delete) but the data return by response is just empty. It looks to me like we are hitting either major API change or some problems with gog servers being under heavy load. There are more details on gog forum: https://www.gog.com/forum/general/gogrepopy_python_script_for_regularly_backing_up_your_purchased_gog_collection_for_full_offline_e/post2887 |
|
Sooo, does anybody now when we can get a fix or how to fix by ourselfes for this mess? |
|
Had some severe weather here last week, so I'm running behind schedule on
several projects. This is still on my to do list but I'm guessing it'll be
late this week-ish before I get to it.
…On Mon, 7 Mar 2022, 5:53 pm GenericJohnDoeNO0, ***@***.***> wrote:
Sooo, does anybody now when we can get a fix or how to fix by ourselfes
for this mess?
—
Reply to this email directly, view it on GitHub
<#53 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABKZ33ZDJU4XJIPCAVFNM7DU6WYYZANCNFSM5OPD662A>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
No stress, take your time! ;) |
|
Reopening this, until it's verified the issue is fully fixed, thanks to the Pull Request. I'm doing an update now, so we'll see if any errors pop up. |
|
Added some additional fixes for this issue (the GOG servers are timing out when reading some of the md5 xml files) If you see an error of the form There is one other error I'm aware of concerning broken encoding when downloading files but I need to do some more investigation before I fix it / request reports. I have another project I have to get back to now, but I'll try and get back to this later this week. |
|
Dying light still throwing errors, will wait now for 24 hours but this problem was still over the last days persistent. https://drive.google.com/file/d/1xZBg4eSgN0Z-AVjM6-Ldm8QCe4OBDUIJ/view?usp=sharing |
|
Hi, Thank you very much for this nice tool! As you have asked - I'm still getting the xml parsing error for several games: elex_ii the_witcher_3_wild_hunt_game worms_wmd_game cyberpunk_2077_game I'll now also wait for 24 hours... |
|
Okay, pushed an update that should fully restore parity with the situation before the GOG updates (ie it should work as well as the website), let me know if any downloads are still broken in exciting ways. If there aren't any of those, then I'll close this up and open a couple of additional topics for reporting stuff that's still broken on GOG's side to get them to fix, one for stuff that works through the website (like most of the md5 xml errors that the last 2 commits fixed) and one for stuff that's just completely broken. I've updated the logging/display a bit to more clearly indicate if an exception is handled / unhandled (a line of text before the exception dumping indicating if the exception was handled or not) and a line indicating when the dump is finished. Hopefully this should make it clearer which errors aren't a severe problem An exception without the wrapper is probably something extremely bad. I'll probably reduce the number of things that actually log detailed exceptions in the future, once we're clearer on the status of stuff. The list of games that I've identified that have issues with their md5 data are a_plague_tale_innocence And something of a special case Which is completely missing actual files again, and those files include the installers. graveyard_keeper is also missing a file but it's only a patch file from a very old version to a less old version that's still too old to continue the patch chain actually get up to date, so I'm not going to lose sleep over it. |
|
Unfortunatelly still many xml errors, i add a full logfile for a complete manifest generation from today. https://drive.google.com/file/d/132OzezuoocS8SQcF8QwgKLcX5nARhYLP/view?usp=sharing Manifest was created with command "python gogrepoc.py update -os windows -lang de en -updateonly" |
|
From a quick look all those errors seem to be handled* (they are md5 xml
data errors that the last couple of commits addressed), so while it's good
to know about them so I can report the issue to GOG they shouldn't actually
be interfering with you adding the actual file to the manifest or
downloading things anymore (it'll just be missing the md5 data).
*Except for the 403 Forbidden ones, but I think those may be files that
aren't actually available for download (it looks like those are pre-order
items or special bonus files that never got uploaded from the link form),
you'd have to find the item with the corresponding link on the website in
the Dying Light game and check that they are downloadable.
…On Thu, Mar 10, 2022 at 10:49 PM GenericJohnDoeNO0 ***@***.***> wrote:
Unfortunatelly still many xml errors, i add a full logfile for a complete
manifest generation from today.
https://drive.google.com/file/d/132OzezuoocS8SQcF8QwgKLcX5nARhYLP/view?usp=sharing
—
Reply to this email directly, view it on GitHub
<#53 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABKZ33YTMH7CYPXMWGLCK3TU7HVU3ANCNFSM5OPD662A>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you modified the open/close state.Message
ID: ***@***.***>
|
|
ok, will have a look but now the script has to shove 300 gb to my external hdd. ;) |
|
After a quick Check: I assume - for dying light - this are dlc which are not downloadable because there are no offlineinstallers for them (the skin packs i think). They are not on the downloadpage of my library and after an offlineinstallation gog galaxy "updates" the game with the missing dlcs. After a quick check in gog galaxy: Here are all dlcs listed with their offline installers.... It seems that the homepage library is fucked up for the browser side.... Or it is something with the "Enhanced Edition": Crash Test Skin Pack are listed with "0 Bytes" and i assume this is the content included in the Enhanced Edition Installer. |
|
Here is what I found from a full update on my library: Those unexpected connection errors cause a huge traceback which makes reading the log extremely difficult at the moement. How do we know that everyhing is handled correctly? |
|
The surest way to make sure something was handled correctly is to
1) check the !info.txt in the corresponding folder and verify that there's
an entry for the file / link and that it's got a proper filename specified
(this indicates the update command worked)
2) let a download command run (either generally or limited to the specific
game id) and check that it either gives a pass for the existing file or
correctly downloads the new file. (Which verifies that the local file is
good)
If you're concerned primarily with the logs though:
Anything that's
A short description of a problem, followed by "The handled exception
was:" , then an exception trackball followed by "End exception report"
should be fine. The traceback is just extra info that might be helpful to
me to understand a problem. I'll probably pair that back once I'm sure
that all the errors are identified and handled properly. I'm logging more
thoroughly at the moment just to verify that all the md5 data XML issues
are fixed and that I'm not accidentally caching unintended errors.
…On Fri, 11 Mar 2022, 2:28 am pasbec, ***@***.***> wrote:
Here is what I found from a full update on my library:
cyberpunk_2077_game
- xml parsing error occurred trying to get md5 data for patch_cyberpunk_2077_Hotfix_1.5_(53778)_to_Build_3864499Change_4719897_(53890).exe
- xml parsing error occurred trying to get md5 data for patch_cyberpunk_2077_1.5_(53680)_to_Hotfix_1.5_(53778)-1.bin
- xml parsing error occurred trying to get md5 data for patch_cyberpunk_2077_1.5_(53680)_to_Hotfix_1.5_(53778)-2.bin
- unexpected connection error fetching md5 data for patch_cyberpunk_2077_1.31_(50000)_to_Build_3836782Change_4719897_dllfix_(53680)-1.bin
- xml parsing error occurred trying to get md5 data for patch_cyberpunk_2077_1.31_(50000)_to_Build_3836782Change_4719897_dllfix_(53680)-4.bin
- xml parsing error occurred trying to get md5 data for patch_cyberpunk_2077_1.31_(50000)_to_Build_3836782Change_4719897_dllfix_(53680)-5.bin
- unexpected connection error fetching md5 data for patch_cyberpunk_2077_1.31_(50000)_to_Build_3836782Change_4719897_dllfix_(53680)-11.bin
elex_ii
- xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048).exe
- xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-1.bin
- xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-2.bin
- xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-3.bin
- unexpected connection error fetching md5 data for setup_elex_ii_1.00b_(54048)-4.bin
- xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-5.bin
- xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-6.bin
-> request failed: ('Connection aborted.', BadStatusLine('п\x93=b¡rc ·\x00öbp\\Êl\x1eìÕÆ\x9b.7?F>>\x967"X=£\x0f\r<¬*ò\x1cõÙ\x02ô%¼fvx\x1a¦¤aô+Ê\x7fL}|,\x0b"\x90KåC\x07Ï$ëwp\tÏK\x1a\x10×9½&k6¥\x8czë«1?\x06?~,K\x07\x8f±!Çtèà±ÝW¥9\x8a\x89e©6yU¼ë\x1cÐZÙÑsýNÂ\x11\x8feADR&\x1c:xrÀ\x07\n'))
- xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-7.bin
- xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-8.bin
- xml parsing error occurred trying to get md5 data for setup_elex_ii_1.00b_(54048)-9.bin
grim_dawn
- unexpected connection error fetching md5 data for setup_grim_dawn_-_ashes_of_malmouth_1.1.9.5_(53749).exe
kingdom_come_deliverance
- xml parsing error occurred trying to get md5 data for setup_kingdom_come_deliverance_-_hd_sound_pack_1.9.6-404-504u_(43503).exe
- unexpected connection error fetching md5 data for setup_kingdom_come_deliverance__band_of_bastards_1.9.6-404-504u_(43503).exe
- xml parsing error occurred trying to get md5 data for setup_kingdom_come_deliverance_-_hd_voice_pack_-_english_1.9.6-404-504u_(43503)-1.bin
metro_exodus
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433).exe
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-1.bin
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-5.bin
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-6.bin
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-7.bin
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-8.bin
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-9.bin
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-10.bin
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-11.bin
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-12.bin
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-13.bin
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-14.bin
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-15.bin
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-16.bin
- xml parsing error occurred trying to get md5 data for setup_metro_exodus_1.0.7.16_(64bit)_(38433)-17.bin
- failed to fetch https://www.gog.com/downloads/metro_exodus/85424
-> request failed: 403 Client Error: Forbidden for url: https://content-system.gog.com/products/1407287452/users/48029035928056325/namespaces/website/download?path=%2Fsecurex&token=eyJhbGciOiJIUzI1NiIsImV4cCI6IjE2NDcwMTMyOTcifQ.eyJjb250ZW50Ijoie1widXNlcl9pZFwiOjQ4MDI5MDM1OTI4MDU2MzI1LFwicHJvZHVjdF9pZFwiOjE0MDcyODc0NTIsXCJwYXRoXCI6XCJcXFwvc2VjdXJleFwifSIsImV4cCI6IjE2NDcwMTMyOTcifQ.er8Ve_3N4DKXDLUAVK-c3Sq3PgtwC09cGRmEIm97FgI&access_token=0DccoptSnwI-MD2WyFmrpM8hFjqAr8I8863u6_KytLRQyyQ7l6WyIZPFHraaR6fm2A1FStWuK-3K_fHGic_FkOwY7HaULAPYu5K-boyXJ9BhKfhHGXqnr5fb9Mx6eI5E78nQABhlGXnuSzOmpkr1xQ. will not retry.
metro_last_light_redux
- unexpected connection error fetching md5 data for setup_metro_last_light_redux_2.0.0.2-2.bin
star_wars_knights_of_the_old_republic
- unexpected connection error fetching md5 data for setup_star_wars_-_knights_of_the_old_republic_1.03_(29871)-1.bin
- xml parsing error occurred trying to get md5 data for setup_star_wars_-_knights_of_the_old_republic_1.03_(german)_(29871).exe
- xml parsing error occurred trying to get md5 data for setup_star_wars_-_knights_of_the_old_republic_1.03_(german)_(29871)-1.bin
star_wars_knights_of_the_old_republic_ii_the_sith_lords
- xml parsing error occurred trying to get md5 data for setup_star_wars_-_knights_of_the_old_republic_ii_1.0b_(29869).exe
- xml parsing error occurred trying to get md5 data for setup_star_wars_-_knights_of_the_old_republic_ii_1.0b_(29869)-1.bin
- xml parsing error occurred trying to get md5 data for setup_star_wars_-_knights_of_the_old_republic_ii_1.0b_(german)_(29869)-1.bin
the_witcher_3_wild_hunt_game
- xml parsing error occurred trying to get md5 data for patch_the_witcher_3_wild_hunt_1.31_(A)_to_1.32.exe
- xml parsing error occurred trying to get md5 data for setup_the_witcher3_exp1_2.0.0.52.exe
- xml parsing error occurred trying to get md5 data for setup_the_witcher3_exp1_2.0.0.52-1.bin
the_witcher_3_wild_hunt_game_of_the_year_edition_game
- no md5 data found for patch_witcher3_de_goty_2.0.0.51.exe
-> request failed: 404 Client Error: Not Found for url: https://gog-cdn-lumen.secure2.footprint.net/token=nva=1646928686~dirs=4~token=0c690a551c73fb1366e67/secure/witcher_3/goty/lang/patch_witcher3_de_goty_2.0.0.51.exe.xml. will not retry.
- no md5 data found for patch_witcher3_de_goty_2.0.0.51-1.bin
-> request failed: 404 Client Error: Not Found for url: https://gog-cdn-lumen.secure2.footprint.net/token=nva=1646928689~dirs=4~token=07142534667750ef7a7b0/secure/witcher_3/goty/lang/patch_witcher3_de_goty_2.0.0.51-1.bin.xml. will not retry.
- no md5 data found for patch_witcher3_de_goty_2.0.0.51-2.bin
-> request failed: 404 Client Error: Not Found for url: https://gog-cdn-lumen.secure2.footprint.net/token=nva=1646928690~dirs=4~token=0bd5c0dbabe10e75fba34/secure/witcher_3/goty/lang/patch_witcher3_de_goty_2.0.0.51-2.bin.xml. will not retry.
- no md5 data found for patch_witcher3_de_goty_2.0.0.51-3.bin
-> request failed: 404 Client Error: Not Found for url: https://gog-cdn-lumen.secure2.footprint.net/token=nva=1646928691~dirs=4~token=069a8164706782d282845/secure/witcher_3/goty/lang/patch_witcher3_de_goty_2.0.0.51-3.bin.xml. will not retry.
victor_vran
- xml parsing error occurred trying to get md5 data for setup_victor_vran_2.07(20190131)_(27183).exe
- xml parsing error occurred trying to get md5 data for setup_victor_vran_2.07(20190131)_(27183)-1.bin
- xml parsing error occurred trying to get md5 data for setup_victor_vran_2.07(20190131)_(27183)-2.bin
walking_dead_a_new_frontier_the
- xml parsing error occurred trying to get md5 data for setup_the_walking_dead_a_new_frontier_patch_1_(64bit)_(20923)-2.bin
worms_wmd_game
- unexpected connection error fetching md5 data for setup_worms_w.m.d_1.0.0.273_(a)_(49738)-1.bin
- xml parsing error occurred trying to get md5 data for setup_worms_w.m.d._all_stars_pack_1.0.0.273_(a)_(49738).exe
xcom2
- xml parsing error occurred trying to get md5 data for setup_xcom2_war_of_the_chosen_374751_(64bit)_(35526)-5.bin
- xml parsing error occurred trying to get md5 data for setup_xcom2_war_of_the_chosen_374751_(64bit)_(35526)-8.bin
- xml parsing error occurred trying to get md5 data for setup_xcom2_war_of_the_chosen_374751_(64bit)_(35526)-9.bin
- unexpected connection error fetching md5 data for setup_xcom2_war_of_the_chosen_374751_(64bit)_(35526)-10.bin
- xml parsing error occurred trying to get md5 data for setup_xcom2_resistance_warrior_pack_374751_(64bit)_(35526).exe
Those unexpected connection errors cause a huge traceback which makes
reading the log extremely difficult at the moement.
How do we know that everyhing is handled correctly?
—
Reply to this email directly, view it on GitHub
<#53 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABKZ334LX5ENEHVM5BXXCPDU7IPL3ANCNFSM5OPD662A>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you modified the open/close state.Message
ID: ***@***.***>
|
Thank you very much for the clarification. I very much appreciate your tool and your active support! |
|
Thumbs up for your work Kalanyr! |
Tried to update today my offline games… to meet with following errors
Did gog change something on their end or is it temporary?
The text was updated successfully, but these errors were encountered: