File not found by id during indexing

[dwaxweiler]

Describe the bug

During the automatic and manual indexing, one file is not found, which I can see in the logs and on the console when running the command manually:

Failed to index folder /user1/files/filename: File by id 1714707 not found

Steps To Reproduce

Waiting for automatic or manual indexing results. It also happens with limiting the scan to the user in question:

php occ memories:index --user user1

Running php occ files:scan user1 did not help.

Platform

- OS: Debian GNU/Linux 11 (bullseye)
- Memories Version: 7.4.1
- Nextcloud Version: 29.0.9
- PHP Version: 8.2.26

Screenshots

No response

Additional context

No response

[MrRinkana]

If you browse the data folder yourself, can you find the file?

What are the permissions of the file (ls -la)?

[dwaxweiler]

If you browse the data folder yourself, can you find the file?

No, this file does not exist.

[MrRinkana]

Then this issue is about stale index entries not being removed.

Do you know how the file got removed? Was/is it added to an album? Info about its lifetime can help the debugging.

External storage (app) or nextcloud internal?

Your issue should be resolved if you run occ memories:index --user user1 --force

But if it does not bother you too much, you could let it be for a while to try to hunt down a cause or steps to reproduce.

[dwaxweiler]

Do you know how the file got removed? Was/is it added to an album? Info about its lifetime can help the debugging.

No, unfortunately, I don't know. We don't use the album feature; The Photos app is deactivated.

Suspicious is that this file has no file extension in the log file, but it as a dot a bit earlier in the name: /user1/files/Recrutement chargé.e de mission juin 2024 Could this have caused an issue somewhere?

External storage (app) or nextcloud internal?

This file path points to the internal folder of a user, and we are also using group folders. I could imagine that this file was moved from the personal to a group folder.

Your issue should be resolved if you run occ memories:index --user user1 --force

Unfortunately, it does not. It ends with the same error message.

[dwaxweiler]

I found the entry with the corresponding fileid in the table oc_filecache but under a group folder path (different one than in the logs). No entry with the same fileid exists in oc_memories. Is there another table I could check?

[MrRinkana]

Very weird indeed. I am not familiar with the database tables memories uses, but maybe you could check if you can find the same etag?

Just to be clear, the "Recrutement chargé.e de mission juin 2024" is an image? Does the file have a proper fileextension at the end?

Have you managed to recreate the issue with a different file? Say if you move a by memories indexed image to a group folder? If you change the name to something similar to "Recrutement chargé.e de mission juin 2024" of a image?

I have just tried to:

1. Rename a file to "Recrutement chargé.e de mission juin 2024.JPG" the file was previously indexed by memories (visible in timeline)
2. Move the file trough the files app to a group folder

But I cant recreate the issue, the file still shows up in memories (at the new location) no errors are logged. I have also tried the same but without the JPG, still works. I moved to a group folder which was then removed from the account, still no error..

I don't have more ideas on how to reproduce the issue right now at least, but I believe that should be the priority, at least as-long as the only issue is the error message (and nothing else is broken).

Edit: forgot to ask: You mention its in the logs, does the file still exist?

[dwaxweiler]

Thanks for looking into this!

I am not familiar with the database tables memories uses, but maybe you could check if you can find the same etag?

Me neither. :) The table oc_memories does not seem to use the etag, or should I check a different table?

Just to be clear, the "Recrutement chargé.e de mission juin 2024" is an image? Does the file have a proper fileextension at the end?

No, it is a directory. By consequence, it has no file extension.

Edit: forgot to ask: You mention its in the logs, does the file still exist?

No, the directory does not exist at that place. Therefore, I want to find in the table where this reference is stored.

[dwaxweiler]

I tried something more. I ran occ files:get 1714707 and got 1714707 is a directory. Then I ran occ files:delete 1714707

Warning: the provided file is accessible by more than one user
  all of the following users will lose access to the file when deleted:

userA:
  - /userA/files/Projects/subfolder/subfolder/Recrutement chargé.e de mission juin 2024
userB:
  - /userB/files/Projects/subfolder/subfolder/Recrutement chargé.e de mission juin 2024
[more users with access to that group folder...]

Delete /userA/files/Projects/subfolder/subfolder/Recrutement chargé.e de mission juin 2024 and all it's contents? [y/N] N

Projects is a group folder, so this warning message makes sense, but the initial error message does not include this group folder path, but directly points to /user/files/Recrutement chargé.e de mission juin 2024, which of course does not exist.