Multithreading support for faster HEIC loading (Microsoft HEIF are super slow in WIndows)

[mdrejhon]

I have a "Camera Uploads" folder of over 10,000 HEIC images synchronized from iPhone/iPads, to multiple computers (via cloud sync currently configured to preserve HEIC/HEIF instead of autoconvert to JPG).

Microsoft's HEIF extensions are extremely slow (CPU based) and single-threaded:
https://www.microsoft.com/en-us/p/heif-image-extensions/9pmmsr1cgpwg

It works with WinThumbsPreloader, but it is ASTONISHINGLY SUPER PAINFUL SLOW (not your fault, it's Microsoft's since it's similarly painful slow in Explorer too) ....

On some computers (that my can take hours to render HEIC images at 1 image per second on some CPUs -- about 10x to 50x slower than JPG files.

To fix the slowness, I would like to see WinThumbsPreloader go multithreaded, to render .heic and .heif files between 8x to 20x+ faster (depending on available cores/threads of your CPU).

[mdrejhon]

Update: I just discovered one of the forks has multithreaded generation added by him:
https://github.com/inthebrilliantblue/WinThumbsPreloader

But the @inthebrilliantblue installer has an "Update check failed" and does not let me install it (even if I uninstall the bruhov version first).

Is there a way to install the multithreaded version easily without recompiling via Visual Studio? Or can the original author (@bruhov) merge the @inthebrilliantblue multithreaded patch into this build?

The multithreading patch is at inthebrilliantblue#1

[inthebrilliantblue]

So all I added was a cli option to toggle the _multithreaded flag, and the parallel foreach code to do the work. I never did get around to touching the installer or add a context menu to select the MTed option.

I mostly just needed a cli option where I setup a task to run a few times a day for a highly rotating dump folder for photographers. I may be able to change the installer, but its been a hot minute since Ive looked at this project. Also while the MTed code works just fine for me, it hasnt really been tested in other environments so Im not sure how it will work for you.

You can download a compiled version of my MTed version on tag 1.0.2:

https://github.com/inthebrilliantblue/WinThumbsPreloader/tree/v1.0.2

[Kieaer]

@inthebrilliantblue How to make the -m option work?
No popup and not preloaded.

I tried
WinThumbsPreloader.exe -m C:\Users\cloud\Downloads\test
WinThumbsPreloader.exe -r -m C:\Users\cloud\Downloads\test
WinThumbsPreloader.exe -m -r C:\Users\cloud\Downloads\test
WinThumbsPreloader.exe -r C:\Users\cloud\Downloads\test // work
WinThumbsPreloader.exe -rm C:\Users\cloud\Downloads\test
WinThumbsPreloader.exe -mr C:\Users\cloud\Downloads\test
WinThumbsPreloader.exe C:\Users\cloud\Downloads\test // work

[mdrejhon]

@Kieaer @inthebrilliantblue
You're right... I just git pulled that repo, compiled it. It appears to not recognize the -m option nor show it in the startup screen (as in screenshot above).

I wonder if part of the pull request wasn't fully uploaded? (Will dive into the source later)

[inthebrilliantblue]

I created pull request #20 with the multi-threaded patch I did. Apparently I just did a Parallel.ForEach with an absurd amount of threads it can spawn for my servers. Someone might want to bring it down, but C# tends to manage the tasks it spawns with Parallel.ForEach based on the computer it runs on, so it "shouldnt" be a problem.

[inthebrilliantblue]

Here is an example of it running on my desktop using 113 threads.

[inthebrilliantblue]

@inthebrilliantblue How to make the -m option work?
No popup and not preloaded.

I tried
WinThumbsPreloader.exe -m C:\Users\cloud\Downloads\test
WinThumbsPreloader.exe -r -m C:\Users\cloud\Downloads\test
WinThumbsPreloader.exe -m -r C:\Users\cloud\Downloads\test
WinThumbsPreloader.exe -r C:\Users\cloud\Downloads\test // work
WinThumbsPreloader.exe -rm C:\Users\cloud\Downloads\test
WinThumbsPreloader.exe -mr C:\Users\cloud\Downloads\test
WinThumbsPreloader.exe C:\Users\cloud\Downloads\test // work

It runs with this:

WinThumbsPreloader.exe -m -r C:\Users\cloud\Downloads\test

Each option needs to be separated for it to be detected. That could be improved upon. As for why it didnt work for you, Im not sure.

What happens if you just do

WinThumbsPreloader.exe

?

[inthebrilliantblue]

@Kieaer You should see this if you just run the exe with no options, make sure it has the -m option listed in the window.

[mdrejhon]

Hello.

1. I did this:
   git clone https://github.com/inthebrilliantblue/WinThumbsPreloader.git

2. I opened the .sln file in Visual Studio 2017

3. I modified AssemblyInfo.cs to Version 1.0.2 as an additional verification that I'm compiling the correct copy

4. I compiled (easily, no errors, no warnings)

5. I ran WinThumbsPreloader.exe to pop up this dialog below, and the -m option does not show in the dialog which is missing the -m option:

@inthebrilliantblue Are you sure that your git pull request is complete? I think there might be missing code.

Can you test a git pull into a 2nd separate folder on your disk, and verify if there are any missing files / missing changes? Please do a verification by this clean pull request -- and you will see the missing "-m" option suggestive your your repo is incomplete (e.g. missing a file or missing a change) -- i.e. a forgotten file to be comitted.

I also viewed your source code files and there's no occurence of the text modification "-m Multi-Threaded mode" which is another 2nd check that makes me suspect this pull request may be missing code.

I did a grep -R -i "Multi-Threaded mode" to find the location of your dialog change, and zero results appeared (anywhere in the @inthebrilliantblue source code tree).

If so, @inthebrilliantblue, would it be possible to add the missing code to your original github & this pull request? This would be a major speedup for me for a File Explorer limitation with .HEIC / .HEIF files.

[inthebrilliantblue]

Looking at my old code, it looks like the AboutForm.cs did not get updated in the repo like it should have, but all of the actual code to run multi-threaded mode is in the repo. That form missing shouldnt cause any issues, grep for "multithreaded", "_multiThreaded", and "Parallel.ForEach" and see if you get results.

[mdrejhon]

Yeah, I see hits on those greps.

Repeating some tests. I'll report back. I'm gonna do a Debug session. It's rare I'm this eager, but -- Microsoft HEIC is so super painfully slow with File Explorer's single-threaded thumbnails generator. (Hope Windows 11 fixes it)

The other day, I tried testing it on a folder full of .HEIC files and it wasn't any faster, attempting to breakpoint on ForEach and seeing what happens.

[inthebrilliantblue]

If your .HEIC files are on spinning HDD's, then it most likely wont be faster. IO will always be the bottleneck, not the actual generation of the thumbnails.

[mdrejhon]

It's definitely not disk.

(1) Samsung 960 Pro SSD here, in a 4-lane NVMe M.2 slot.

and

(2) Larger 2+ megabyte .HEIC files take almost 1 second to convert even on an SSD, while even larger 3-5 megabyte .JPG files can convert 10x+ faster even for HDD. .HEIC is that painfully slow on Windows machines compared to Mac because Microsoft uses 100% CPU (no GPU acceleration) for converting HEIC.

HEIC is more than 10x slower than JPG on Windows

On Windows machines specifically, it is literally honestly truly more than an order of magnitude slower per byte. It's why I desparately want multithreading for .HEIC because Microsoft didn't optimize .HEIC like Apple did.

On a single thread, it can take HOURS to convert an iPhone's worth of many (tens) thousands of smartphone photos captured natively in Apple's preferred .HEIC / .HEIF formats. That's why most sync software (e.g. Dropbox) converts to JPG first before syncing, but I didn't want the quality loss of downconverting .HEIC to bigger .JPG files when syncing from my iPhone/iPads.

It's faster to cloud-sync thousands of photos to 5-6 computers if the images are smaller file size like HEIC. Many Internet connections have slow upload speeds for cloud syncing of photos, so this is where smaller HEIC file sizes helps. Although I've since upgraded to FTTH with fast upload, the HEIC still remains. I'm just paying a severely time-punitive thumb-generation penalty, however. And this is why I express great interest in multithreaded HEIC thumbnail generation.

Imagine maximizing your File Explorer window in Large Thumbnails mode, and having to wait 30-60 seconds for all the thumbnails to generate -- just only for that screenful. PgUp and PgDn is simply AGONIZING due to Microsoft's suboptimal HEIC engine

For People Unfamiliar With Why HEIC is Good

800 kilobyte HEIC photos look better than 2 megabyte JPG files -- it's a pretty compact, and efficient photo compression format. I made the (semi-stupid) decision not to pre-convert to JPG for performance, and am having to bear Windows inefficiency with HEIC, for the superlative compression efficiency of HEIC.

(JPEG efficiency harkens way back to the MPEG1 days literally, while HEIC is pretty much H.265 in compression efficiency -- so huge amount of space savings keeping it HEIC native)

If they were all Macs it would be so much faster, since Apple's HEIC implementations are GPU accelerated. Thumbnails for HEIC just flies almost 100x faster on an M1 Mac, for example.

Everytime I dump a new batch of photos, I have to put up with Windows' slowness in regenerating thumbnails on 5-6+ sync'd computers, as they keep separate thumbnails databases.

[inthebrilliantblue]

Since I do not have any native HEIC files I cant test it. Doing a conversion from jpg to heic probably wouldnt be a good test either. Though after converting one myself to see if my W10 could make a thumbnail, it couldnt.

Looking around, I see that the Windows store has something that could, but Im not going to pay money for that. People seem to be suggesting CopyTrans HEIC (https://www.copytrans.net/copytransheic/) for thumbnail generation. Is this what you are using? It seems to be pretty fast to me, but again I dont have native HEIC files to test this against.

Edit: HOLY COW the space savings batman! I just did a random folder of 42MP pictures I have, and dang does it do a good job of saving space compared to jpg. I can see why people use it!

[mdrejhon]

No, I am using the Microsoft HEIF Image Extensions:

Microsoft HEIF Extensions (free)
https://www.microsoft.com/en-us/p/heif-image-extensions/9pmmsr1cgpwg?activetab=pivot:overviewtab

I also had to install this concurrently too (99 cents), for more universal HEIF / HEIC support.

Microsoft HEVC Extensions (99 cents)
https://www.microsoft.com/en-us/p/hevc-video-extensions/9nmzlz57r3t7?activetab=pivot:overviewtab

Once I had both, I have universal thumbnail support in Windows File Explorer for all native iDevice file formats. That said, it is very painfully slow in refreshing thumbnails in Explorer due to lack of Microsoft performance optimizations.

On some computers thumbnails generate at about 2-3 images per second, on a different computer of mine, it is 1 image per second. It clearly appears to be dependant on the CPU thread performance -- far more so than HDD performance.

All my Apple iDevices save straight to HEIC, with no conversion, so syncing in their original format means no quality loss from syncing. It's nice to see 800 kilobyte HECs better quality than 2 megabyte JPG files. If your JPG came from Apple devices, then you're wasting image quality -- configure your sync software (iCloud, Dropbox, whatever) to keep them original format which is HEIC.

The alternative file format I am considering was AVIF, which upcoming web browsers are going to support (Microsoft Windows 11 supports AVIF and Edge will get support). It's roughly similar efficiency, and probably will be more well supported by Microsoft as a popular next-generation post-JPG file format. Keep an eye on AVIF and HEIC/HEIF as jockeying next generation post-JPG file formats that also works in web browsers -- the cards are showing that at least one of these next generation image file formats are going to finally become popular cross-platform.

[inthebrilliantblue]

So doing a quick conversion of the project to a console project and some edits to try catches, Im starting to see this error that the original code for ThumbnailPreloader.cs was ignoring:

System.Runtime.InteropServices.COMException (0x8004B200): Exception from HRESULT: 0x8004B200
   at ConsoleWTP.ThumbnailPreloader.IThumbnailCache.GetThumbnail(IShellItem pShellItem, UInt32 cxyRequestedThumbSize, WTS_FLAGS flags, ISharedBitmap& ppvThumb, WTS_CACHEFLAGS& pOutFlags, WTS_THUMBNAILID& pThumbnailID)
   at ConsoleWTP.ThumbnailPreloader.PreloadThumbnail(String filePath) in C:\Users\scott\Downloads\WinThumbsPreloader-master (1)\WinThumbsPreloader-master\WinThumbsPreloader\ConsoleWTP\ThumbnailPreloader.cs:line 28

Im not sure what this means, I will need to research it as this class is not one I messed with yet.

[inthebrilliantblue]

Ok I think that error is because I have some 100MiB png files that I converted to HEIC files, and imagemagick didnt do a good job converting them.

I have pushed a simplified console version of WTP to my repo called ConsoleWTP.

To use, just call "ConsoleWTP.exe path\to\directory". It will do subdirectories and multithreading. Currently there is no way to turn those two options off since I did a quick copy paste of the code. Try it and see what it does. It will take a moment to load up the threads and display the files it is currently working on.

[mdrejhon]

Googling the HRESULT shows this:

"Thumbnail Cache: The Shell item does not support thumbnail extraction. For example, .exe or .lnk items."

https://www.hresult.info/FACILITY_ITF/0x8004B200

[mdrejhon]

Code review issue (of the Windows GUI version)

Another bug that I think I have now discovered is that while I can get multithreaded working with a folder of JPG files (not HEIC), I can't do it via the right-click menu by adding the -m option to the first context menu by manual regedit:

(I replaced the executable with my "v1.0.2" compiled version)

So that's 2 bugs (possibly 1 Microsoft fault) holding back from convenient speed-thumbnailing HEIC/HEIF files:

• Works by command line at the CMD prompt, but not via editing right click context menu;
• HEIC/HEIF files doesn't seem to like being multithreaded somehow (!?)

Strange for the 2nd issue. Hopefully it's just a simple permissions issue or something?

[mdrejhon]

Update for item 2, I edited

new ParallelOptions { MaxDegreeOfParallelism = 2048 },

into:

new ParallelOptions { MaxDegreeOfParallelism = 1 },

And it works again but agonizingly slowly. As soon as I change it to 2 or more threads, HEIC/HEIF files fails. So I'm now increasingly convinced it might be (theoretically) a Microsoft limitation of the HEIF Extensions. (And not any other issue such as thread permission issue specific to HEIF etc).

This would be utterly supremely annoying if this is true. (unfixable Microsoft-created performance restriction)

Maybe I should try the other paid HEIF to see if it is more "compatible". (I have already purchased it, but found no speed difference on my side -- but it might be more multithreading friendly)

[Kieaer]

@inthebrilliantblue

[inthebrilliantblue]

Update for item 2, I edited

new ParallelOptions { MaxDegreeOfParallelism = 2048 },

into:

new ParallelOptions { MaxDegreeOfParallelism = 1 },

And it works again but agonizingly slowly. As soon as I change it to 2 or more threads, HEIC/HEIF files fails. So I'm now increasingly convinced it might be (theoretically) a Microsoft limitation of the HEIF Extensions. (And not any other issue such as thread permission issue specific to HEIF etc).

This would be utterly supremely annoying if this is true. (unfixable Microsoft-created performance restriction)

Maybe I should try the other paid HEIF to see if it is more "compatible". (I have already purchased it, but found no speed difference on my side -- but it might be more multithreading friendly)

Ive used the CopytransHEIC home version, and it seems to work fairly ok, but it still fails to generate thumbnails for some files. Im not sure there is much we can do to fix that. Id be interested to know if W11 has any improvements that make this a non-issue.

@inthebrilliantblue

@Kieaer Yes, I am aware of this. Removing the -m option makes it work, while having the -m option makes it crash. After trying it on jpgs, pngs, and heic files, it only does this on the heic files. We are trying to figure out if this is a WTP problem or a Windows problem.

[mdrejhon]

Any new discoveries on this under Windows 11?

Thousands of heifers have exited the barn and are grazing throughout my computer, with these stubborn HEIFers not wanting to hurry up. Moo. [/pun]

Deciding whether or not to upgrade to Windows 11. If I knew this was fixed under Windows 11, I'll upgrade tomorrow.

[arturdd]

Hello everyone,
Those of you who need a multi-core preloading of HEIC files, please take a look on my fork.
As Microsoft HEIF Extensions appears not to be thread safe, I've added a workaround - A copy if this application is started for each .HEIC file in the background. This way it's quite fast. I've tested only a console version

[mdrejhon]

Fantastic!
So the very fast HEIC implementation is in the fork https://github.com/arturdd/WinThumbsPreloader ?

Nice, a @arturdd fork of @inthebrilliantblue fork of @bruhov original.

[mdrejhon]

@bruhov I compiled the multithreaded-HEIC version, and am having an exception when I try to run it:

'WinThumbsPreloader.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll'. Symbols loaded.
'WinThumbsPreloader.exe' (CLR v4.0.30319: DefaultDomain): Loaded 'C:\Projects\Tools\WinThumbsPreloader\WinThumbsPreloader\WinThumbsPreloader\bin\Debug\WinThumbsPreloader.exe'. Symbols loaded.
'WinThumbsPreloader.exe' (CLR v4.0.30319: WinThumbsPreloader.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Windows.Forms\v4.0_4.0.0.0__b77a5c561934e089\System.Windows.Forms.dll'. Cannot find or open the PDB file.
'WinThumbsPreloader.exe' (CLR v4.0.30319: WinThumbsPreloader.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll'. Cannot find or open the PDB file.
'WinThumbsPreloader.exe' (CLR v4.0.30319: WinThumbsPreloader.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Drawing\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Drawing.dll'. Cannot find or open the PDB file.
'WinThumbsPreloader.exe' (CLR v4.0.30319: WinThumbsPreloader.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll'. Cannot find or open the PDB file.
'WinThumbsPreloader.exe' (CLR v4.0.30319: WinThumbsPreloader.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Configuration\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Configuration.dll'. Cannot find or open the PDB file.
'WinThumbsPreloader.exe' (CLR v4.0.30319: WinThumbsPreloader.exe): Loaded 'C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System.Xml\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.dll'. Cannot find or open the PDB file.
Exception thrown: 'System.IO.DirectoryNotFoundException' in mscorlib.dll
Exception thrown: 'System.IO.DirectoryNotFoundException' in mscorlib.dll
Exception thrown: 'System.IO.DirectoryNotFoundException' in mscorlib.dll
An unhandled exception of type 'System.IO.DirectoryNotFoundException' occurred in mscorlib.dll
Could not find a part of the path 'C:\Projects\Tools\WinThumbsPreloader\WinThumbsPreloader\WinThumbsPreloader\bin\Debug\path\to\dir'.


The thread 0xc3f4 has exited with code 0 (0x0).
Unhandled Exception: System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\Projects\Tools\WinThumbsPreloader\WinThumbsPreloader\WinThumbsPreloader\bin\Debug\path\to\dir'.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileSystemEnumerableIterator`1.CommonInit()
   at System.IO.FileSystemEnumerableIterator`1..ctor(String path, String originalUserPath, String searchPattern, SearchOption searchOption, SearchResultHandler`1 resultHandler, Boolean checkHost)
   at System.IO.Directory.GetFiles(String path, String searchPattern, SearchOption searchOption)
   at WinThumbsPreloader.DirectoryScanner.GetItemsBulk() in C:\Projects\Tools\WinThumbsPreloader\WinThumbsPreloader\WinThumbsPreloader\DirectoryScanner.cs:line 39
   at WinThumbsPreloader.ThumbnailsPreloader.<Run>b__15_0() in C:\Projects\Tools\WinThumbsPreloader\WinThumbsPreloader\WinThumbsPreloader\ThumbnailsPreloader.cs:line 161
   at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at WinThumbsPreloader.ThumbnailsPreloader.<Run>d__15.MoveNext() in C:\Projects\Tools\WinThumbsPreloader\WinThumbsPreloader\WinThumbsPreloader\ThumbnailsPreloader.cs:line 143
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
The program '[22192] WinThumbsPreloader.exe' has exited with code 0 (0x0).

[mdrejhon]

OK, @arturdd -- Line 161 of ThumbnailsPreloader.cs is now calling a hitherto formerly unused function called GetItemsBulk (in DirectoryScanner.cs) that seems to have a placeholder path "path/to/dir"

It appears maybe your git commit might be incomplete -- e.g. you're using a working version but didn't check in final changes yet?

(Theory)

[mdrejhon]

UPDATE on @arturdd speedy HEIC generator -- I changed the code inside GetItemsBulk() from "path/to/dir" to the path class member and it finally worked. HEIC files do generate 4-8x faster now!

But there's a bug -- it only focuses on HEIC and doesn't pregenerate PNG / JPG / MOV thumbnails. Something's odd -- maybe the logic that sped up HEIC generation doesn't work on non-HEIC files?

[mdrejhon]

@arturdd Are you able to commit any bug fixes? Thanks!

[arturdd]

@arturdd Are you able to commit any bug fixes? Thanks!

I am sorry for that. I've fixed my last commit & double checked it. Now it's OK

By default, multi-threading is disabled. To preload thumbnails using multi CPU cores, remember to add "-m" as a parameter to the executable when launching as console
or
change the line 28 of Options.cs to "multithreaded = true;" and rebuild solution.

[arturdd]

But there's a bug -- it only focuses on HEIC and doesn't pregenerate PNG / JPG / MOV thumbnails. Something's odd -- maybe the logic that sped up HEIC generation doesn't work on non-HEIC files?

That's interesting. I've tested this build on ~4000 of .jpg files and ~7000 .heic files and it worked well.
I can confirm that on my system, thumbnails for .mp4 files were not pre-generated during multithreaded (parallel) pass.

I'll do some tests, hopefully I'll find a fix

[mdrejhon]

Thank you! Keep fixing those "not HEIC" file issues,

Still, that forked modified version of WinThumbsPreloader is currently the world's fastest HEIC/HEIF generator for Windows PCs --

I have not seen anything generate HEIC files faster than the modified HEIC-compatible arturdd fork of WinThumbsPreloader. Seeing HEIC generate thumbnails for over 50+ HEIC files a second on some heavily multicore-CPU systems such as Threadripper.

www.github.com/arturdd/WinThumbsPreloader

More than 50 HEIC files a second on a Threadripper CPU!

@microsoft should just get with the program and fix their abysmally slow software-based single-threaded HEIC engine that often generates only 1 to 2 thumbnails a second -- but for now this is a great workaround for folders containing thousands of HEIC's synced verbatim (JPG conversion disabled) from multiple iPhones/iPads.

[ErikLtz]

Just want to add this is a really great utility - and as "it should have been"! Installed in Win11, right clicked folder structure with about 3 000 synced HEIC photos (6 GB) from Android phone and using about 400 threads (!) all thumbnails where recursively created on my laptop (8C/16T i7 10875) in just a few minutes. Really great and I no longer consider switching back to JPG for faster thumbnails in Windows at least 😊

[maleko48]

Any ideas why arturrd's version 1.0.3 ( http://www.github.com/arturdd/WinThumbsPreloader ) processes all of the images successfully on my wife's laptop (Win10x64 w/i7 3632QM & Samsung SATA SSD) but upon entering the folders in Windows Explorer, the HEIC icons are still visible and continue to slowly render thumbnails at about 1 per second (same as before)?

All of her other file preview types seem to be instantly rendered upon subsequent entry into the folders, as expected, but not the HEIC files specifically which was the whole reason I started down this path to help her with her iPhone photo dumps folders containing a few thousand images per folder roughly.

[carlosvsilva]

I tried arturrd's version 1.0.3 ( http://www.github.com/arturdd/WinThumbsPreloader ) on Windows 11 22H2 Version 10.0.22621.232 (Release Preview) and it crashes and exits in the middle of the process.
I found a .dmp memory dump file which I'm attaching here, as arturdd github project does not have "Issues" tab available for this fork. Is a fix or workaround possible? Thanks
WinThumbsPreloader.exe.20176.zip

[mdrejhon]

Question:
What is Windows 11's status on HEIC performance? Is it any faster?

[carlosvsilva]

Dont notice any speed differences or issues. Im not using this for the "speed", just to workaround another project bug rendering the thumbnails on my PC.

[kidzgy]

But there's a bug -- it only focuses on HEIC and doesn't pregenerate PNG / JPG / MOV thumbnails. Something's odd -- maybe the logic that sped up HEIC generation doesn't work on non-HEIC files?

That's interesting. I've tested this build on ~4000 of .jpg files and ~7000 .heic files and it worked well. I can confirm that on my system, thumbnails for .mp4 files were not pre-generated during multithreaded (parallel) pass.

I'll do some tests, hopefully I'll find a fix

Thanks a lot for providing such a great software! I found that in version 1.0.3, when using multi-threaded mode to generate thumbnails, the progress window disappears very slowly if you click "X" or "Cancel" in the middle. If you cancel in the single-threaded window, you can see the progress window disappear immediately.

[ririliu]

But there's a bug -- it only focuses on HEIC and doesn't pregenerate PNG / JPG / MOV thumbnails. Something's odd -- maybe the logic that sped up HEIC generation doesn't work on non-HEIC files?

That's interesting. I've tested this build on ~4000 of .jpg files and ~7000 .heic files and it worked well. I can confirm that on my system, thumbnails for .mp4 files were not pre-generated during multithreaded (parallel) pass.

I'll do some tests, hopefully I'll find a fix

@arturdd I got the same problem like mdrejhon have, when add -m parameter, it doesn't generate PNG / JPG thumbnails, when without -m parameter, it can generate PNG / JPG thumbnails

[Devicetron]

version 1.0.3 doesn't create jpg thumbnails on my machine is this normal? is it only for iphone images? maybe it doesn't work anymore on Windows 11? @arturdd @bruhov

[Devicetron]

can someone share me the last multithreaded version that works on jpgs?

[VL4DST3R]

@Devicetron I don't think there is one that works properly. At least i couldn't get it to work no matter what i tried. Arguments are also a pain to work with, as you can see above, they are very poorly documented and you can find conflicting information in other threads here about using them.

I ended up bootleg multithreading it myself using the "classic" version, by starting multiple instances for separate subfolders. Be careful though, its a slippery slope and if you wanna be smart about it (like i did) and do it via a function for every subfolder in your main folder, it can very quickly rack up into hundreds or thousands of processes and bring your whole pc down (and the thumbnail database with it- causing it to reset)

[Devicetron]

yes I feel winthumbspreloader is dead now...

[VL4DST3R]

yes I feel winthumbspreloader is dead now...

if you mean abandonned by its creator then absolutely , but that doesn't mean its not still very useful despite its limitations. It's also (sadly) the only tool of this kind, so unless someone picks it up and finishes it, we'll have to work around its limitations.

[mdrejhon]

Please upvote these two items in Microsoft's Windows 11 Feedback Hub:

.HEIC Thumbnails Generate Slowly
https://aka.ms/AAj029g

EDIT: Microsoft has made that hub item private.

[quinnaissance]

Please upvote these two items in Microsoft's Windows 11 Feedback Hub:

.HEIC Thumbnails Generate Slowly https://aka.ms/AAj029g

Link doesn't seem to work for me. "Your account doesn't have access to this feedback".

[Mfarooq360]

yes I feel winthumbspreloader is dead now...

if you mean abandonned by its creator then absolutely , but that doesn't mean its not still very useful despite its limitations. It's also (sadly) the only tool of this kind, so unless someone picks it up and finishes it, we'll have to work around its limitations.

I've created a fork which should fix this issue, it works with HEIC files and JPEGs now with multithreading.

[kidzgy]

yes I feel winthumbspreloader is dead now...

if you mean abandonned by its creator then absolutely , but that doesn't mean its not still very useful despite its limitations. It's also (sadly) the only tool of this kind, so unless someone picks it up and finishes it, we'll have to work around its limitations.

I've created a fork which should fix this issue, it works with HEIC files and JPEGs now with multithreading.

One question is strange, if I put the keyword "*. *", the thumbnails of the search results still need to be cached, I wonder if there is a way to solve this?

[Mfarooq360]

yes I feel winthumbspreloader is dead now...

if you mean abandonned by its creator then absolutely , but that doesn't mean its not still very useful despite its limitations. It's also (sadly) the only tool of this kind, so unless someone picks it up and finishes it, we'll have to work around its limitations.

I've created a fork which should fix this issue, it works with HEIC files and JPEGs now with multithreading.

One question is strange, if I put the keyword "*. *", the thumbnails of the search results still need to be cached, I wonder if there is a way to solve this?

I'm not sure where to put that keyword into to reproduce the problem, is it in the file name of the images?

[kidzgy]

yes I feel winthumbspreloader is dead now...

if you mean abandonned by its creator then absolutely , but that doesn't mean its not still very useful despite its limitations. It's also (sadly) the only tool of this kind, so unless someone picks it up and finishes it, we'll have to work around its limitations.

I've created a fork which should fix this issue, it works with HEIC files and JPEGs now with multithreading.

One question is strange, if I put the keyword "*. *", the thumbnails of the search results still need to be cached, I wonder if there is a way to solve this?

I'm not sure where to put that keyword into to reproduce the problem, is it in the file name of the images?

Put "*. *" into the search field in Explorer, and set the details view to the oversized icon, you can see that it needs to be re-cached.
I had this problem before with version 1.0.3, but now with version 1.0.4, I found that I don't have this problem anymore. I think it's because version 1.0.3 didn't effectively preload the entire cache.

[Mfarooq360]

I've created a fork which should fix this issue, it works with HEIC files and JPEGs now with multithreading.

One question is strange, if I put the keyword "*. *", the thumbnails of the search results still need to be cached, I wonder if there is a way to solve this?

I'm not sure where to put that keyword into to reproduce the problem, is it in the file name of the images?

Put "*. *" into the search field in Explorer, and set the details view to the oversized icon, you can see that it needs to be re-cached. I had this problem before with version 1.0.3, but now with version 1.0.4, I found that I don't have this problem anymore. I think it's because version 1.0.3 didn't effectively preload the entire cache.

I see, it appears to work now which is surprising considering I only modified a couple lines of code. I've also now fixed the issue of the program not closing in multi-threaded mode. I may release a new version soon, but let me know if there's anything else you want me to try and fix.

[VL4DST3R]

@Mfarooq360 could you maybe make the argument parser more flexible as well? See here.

[Mfarooq360]

@Mfarooq360 could you maybe make the argument parser more flexible as well? See here.

Should be working now, I'll be releasing the next version with these fixes soon.

[Mfarooq360]

@Mfarooq360 could you maybe make the argument parser more flexible as well? See here.

Version 1.0.5 has been released now, allowing the application to close properly in Multi-Threaded mode and supports separate arguments in the command line.

[mdrejhon]

Please upvote these two items in Microsoft's Windows 11 Feedback Hub:
.HEIC Thumbnails Generate Slowly https://aka.ms/AAj029g

Link doesn't seem to work for me. "Your account doesn't have access to this feedback".

Seems like Microsoft made this item private. I'm not 100% sure why. Maybe a duplicate of a different HEIC / HEIF item.

Version 1.0.5 has been released now, allowing the application to close properly in Multi-Threaded mode and supports separate arguments in the command line.

For those coming to this github (e.g. via google search) looking for the best multithreaded HEIC/HEIF thumbnails generator, here's the @Mfarooq360 fork which is now up to v1.1.0:

https://github.com/Mfarooq360/WinThumbsPreloader

[mdrejhon]

Version 2.0.0 Beta 2 Released

Version 2.0.0 of WinThumsPreloader is better, in the @Mfarooq360 fork:

From his comment: Mfarooq360#2 (comment)

Massively more up to date, and massively more HEIC/HEIF reliability and performance than any fork ever, more than @inthebrilliantblue (v1.0.2), @arturdd (v1.0.3) and @bruhov (v1.0.1) versions.

This is important if you need to generate thumbnails for an entire 256GB smartphone full of images, in mere minutes.

@Mfarooq360 = Current 🏆 Fork

As of December 2023, utilizing his test branch, with code commits made as recent as last week. Some other of the 40+ forks of the author's original may get ahead, but THAT is the trophy fork so far, by a wide margin.