open folder / limitation?

[schimschewski]

i'm using very often the "open folder"-feature to export the entire movies in subdirectories e.g.
this was working like a charm in <= version 23.x

since 24.0.1 there seems to be a limitation or some cache problems, it does not show up all results. reopening the folder results in getting some missing files (sometimes only 5 and 10) from the first attempt.
since 24.0.5 it does not show up anything, seems like no recursive search in subdirectories is done.

it this a bug or an expected behaviour? see screenshots attached.

[JeromeMartinez]

it this a bug or an expected behaviour?

Bug, but we can not reproduce it, subdirs are still well scanned on our side
What is the memory usage (through the task manager) during the scan?

By the way, using the command line version may be better for such slow task ;-).

[schimschewski]

it this a bug or an expected behaviour?

Bug, but we can not reproduce it, subdirs are still well scanned on our side What is the memory usage (through the task manager) during the scan?

By the way, using the command line version may be better for such slow task ;-).

You mean the memory usage for the latest version?

Talking for version 24.05.1:
The cpu takes some usage for only 3-4 seconds, and then it goes down.
the memory usage is not affected. even after minutes -> no results shown
Its working for other directories with lower subdirs count very well, but stopped working for higher amount.
the same directory is still working with v23.

thanks for the hint with the cli. will give it a try

[JeromeMartinez]

Its working for other directories with lower subdirs count very well, but stopped working for higher amount.

Could you spot a minimum directory structure that demonstrate such issue?
We would like to be able to reproduce it on our side for easier debugging.

the same directory is still working with v23.

Weird there, we don't see what we could have changed which would have an impact there.

thanks for the hint with the cli. will give it a try

classic usage:
MediaInfo YourDirectory > TargetTextFile.txt
The CLI is available in 64-bit as well as in 32-bit, please test both, it only the 32-bit version fails it may be due to a too high RAM usage (> 2 GiB) and we should track down the source of the RAM usage, else it is something else (but what? That is the question...).

[schimschewski]

Its working for other directories with lower subdirs count very well, but stopped working for higher amount.

Could you spot a minimum directory structure that demonstrate such issue? We would like to be able to reproduce it on our side for easier debugging.

i will try, give me some time.

the same directory is still working with v23.

Weird there, we don't see what we could have changed which would have an impact there.

thanks for the hint with the cli. will give it a try

classic usage: MediaInfo YourDirectory > TargetTextFile.txt The CLI is available in 64-bit as well as in 32-bit, please test both, it only the 32-bit version fails it may be due to a too high RAM usage (> 2 GiB) and we should track down the source of the RAM usage, else it is something else (but what? That is the question...).

just played around with the cli and took some notices from task-manager:

32-bit cli: the task is starting, the memory usage is increasing and the whole task stops @ ~100mb memory usage. the output json is 0byte.

64-bit cli: it works! the memory usage is ~150mb (highest peak), the output json is complete with every entry (as in v23 gui)

[JeromeMartinez]

32-bit cli: the task is starting, the memory usage is increasing [...] 64-bit cli: it works! the memory usage is ~150mb (highest peak)

So something related to 32-bit (GUI still uses the 32-bit build internally, the 64-bit is for the Explorer integration) vs 64-bit but not the total memory usage, fun...

Maybe a file makes MediaInfo to try to temporary allocate a big amount, it would be great if you can find which file(s) are the trigger for the missing output, and then share a use case (email at [email protected] if the link can not be shared publicly).

[schimschewski]

Its working for other directories with lower subdirs count very well, but stopped working for higher amount.

Could you spot a minimum directory structure that demonstrate such issue? We would like to be able to reproduce it on our side for easier debugging.

tried it with:
D:\test
D:\test\Some.Movie.Name
D:\test\Some.Movie.Name\movie.mkv
D:\test\Some.Movie.Name\movie2.mkv

it fails when opening "D:\test". it doesn't show up any result. It seems, like the new dialog is not going recursively through the sub-directories.

it succeeds when opening "D:\test\Some.Movie.Name"

[JeromeMartinez]

it fails when opening "D:\test".

Also with 32-bit CLI?
GUI drag and drop?

like the new dialog is not going recursively

The new dialog provides the same info (a path name) to the underlying library so it should not have any impact, weird.

[schimschewski]

it fails when opening "D:\test".

Also with 32-bit CLI? GUI drag and drop?

test-folder

32-bit-cli: working
gui "open folder"-dialog: fails, nothing is shown
gui "drap&drop": works

original-folder

32-bit-cli: fails as noted above
gui "open folder"-dialog: fails, nothing is shown
gui "drag&drop": works, but not with all results (as in 24.1)

i think there are 2 bugs. the new dialog in 24.5 and the issue that not all results are processed

[JeromeMartinez]

i think there are 2 bugs.

From what you write, it seems that... I can reproduce this issue, I check why.

original-folder
32-bit-cli: fails as noted above

Sorry, still need your help on this one, for knowing which file is creating the issue.

[schimschewski]

i think there are 2 bugs.

From what you write, it seems that... I can reproduce this issue, I check why.

original-folder
32-bit-cli: fails as noted above

Sorry, still need your help on this one, for knowing which file is creating the issue.

i will try to help you.
all my big folders/drive (i have 3) have the same issue. its aborting at some point. and on second attempt it throws some missing file into the result (as there is something in the cache pipeline).

could not figure out if its because of a certain file yet.

[cjee21]

From what you write, it seems that... I can reproduce this issue, I check why.

I can reproduce the subfolder issue. The new dialog returns the path of the selected folder without a trailing backslash and passes it to MediaInfoLib. What happens though is at the check before this part. TDirectory::GetFiles(FolderOpenDialog1->FileName).Length != 0 seems to return zero when there are no files at the root of the selected directory so the rest does not even run at all.

[JeromeMartinez]

The new dialog returns the path of the selected folder without a trailing backslash and passes it to MediaInfoLib

Thanks @cjee21, so seems to be a BCB issues?
In practice MediaInfoLib API can accept a directory, maybe better to provide directly the directory name to it but it owuld be longer to implement (I would have to do it), but maybe a hot fix could be to add the trailing backslash so TDirectory::GetFiles can do its job correctly? Or not enough? (not tested on my side yet)

[cjee21]

The new dialog returns the path of the selected folder without a trailing backslash and passes it to MediaInfoLib

Thanks @cjee21, so seems to be a BCB issues? In practice MediaInfoLib API can accept a directory, maybe better to provide directly the directory name to it but it owuld be longer to implement (I would have to do it), but maybe a hot fix could be to add the trailing backslash so TDirectory::GetFiles can do its job correctly? Or not enough? (not tested on my side yet)

I can make a patch to just check that it returns a non-empty directory path and not care about whether there are actually files in it. Tested that it works on my demo folder with sub-folders and many files.

PS By the way, have you seen my branch for implementing WebView2 (#857 (comment))? Wondering if you want to proceed with that. If so I'll make a PR for that.

[JeromeMartinez]

I can make a patch to just check that it returns a non-empty directory path and not care about whether there are actually files in it.

Please do.

PS

Still on my todo-list.

[cjee21]

Please do.

Done

Still on my todo-list.

Okay, no hurry. Just wondering if you have seen it.

[cjee21]

I cannot reproduce the other bug with my sub-directories that have a total of 90+ files. All of them load after waiting for some time. I don't see any change for Windows GUI in regards to opening folders in version 24.01 or 24.01.1 (v23.11...v24.01.1#diff-ad45e0d3e6311c81d8158055867ab84f3dd36b245a048b283123f0e1b01565bb).

[schimschewski]

I cannot reproduce the other bug with my sub-directories that have a total of 90+ files. All of them load after waiting for some time. I don't see any change for Windows GUI in regards to opening folders in version 24.01 or 24.01.1 (v23.11...v24.01.1#diff-ad45e0d3e6311c81d8158055867ab84f3dd36b245a048b283123f0e1b01565bb).

yeah, i can reproduce it only with folders with 1500+ files.
but its not an entire file or filename. it seems to hit some limits or so.

it's not the folder count or files count.

i will run some more tests

[schimschewski]

I don't see any change for Windows GUI in regards to opening folders in version 24.01 or 24.01.1 (v23.11...v24.01.1#diff-ad45e0d3e6311c81d8158055867ab84f3dd36b245a048b283123f0e1b01565bb).

the bug is also in the cli, not only gui.
the main difference:
cli stops working and the output is a 0-byte-file.
gui shows some weird behaviour. it outputs some results, and on another folder opening the missing results from the first attempt are shown up in 5-/10-steps

[cjee21]

I don't see any change for Windows GUI in regards to opening folders in version 24.01 or 24.01.1 (v23.11...v24.01.1#diff-ad45e0d3e6311c81d8158055867ab84f3dd36b245a048b283123f0e1b01565bb).

the bug is also in the cli, not only gui

Then likely in the MediaInfoLib which I know nothing about.

[JeromeMartinez]

Then likely in the MediaInfoLib which I know nothing about.

On my side.

@schimschewski, this ticket is for the GUI issue and it will be closed when the GUI issue is resolved, please open a MediaInfoLib ticket for the other one common to GUI and CLI.

[JeromeMartinez]

dev snapshot.