Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fileMode-fix for issue #1694 Windows bug #1710

Merged
merged 4 commits into from
Jan 18, 2025

Conversation

Jalileh
Copy link
Contributor

@Jalileh Jalileh commented Jan 10, 2025

windows symlinks are strange and they are not treated as directories in go, so using just fileInfo.Dir isn't enough.
Now treated as directories.

PR branch
image
main branch
image

bug issue link: #1694

windows symlinks are strange and they are not treated as directories in
go, so using just fileInfo.Dir isn't enough.
Copy link
Contributor

coderabbitai bot commented Jan 10, 2025

Walkthrough

The pull request introduces a new constant and function in the utilfn package related to Windows symbolic links. Two constants, winFlagSoftlink and winFlagJunction, are added to represent specific file attributes for symbolic links and junctions. A new function, WinSymlinkDir, is implemented to determine if a given path is a Windows symbolic link or junction by checking the file attributes and file extension. Additionally, the existing DetectMimeType function is modified to utilize WinSymlinkDir, enabling it to return "directory" for both actual directories and Windows symbolic links. These changes improve the detection capabilities of DetectMimeType in Windows environments.

Finishing Touches

  • 📝 Generate Docstrings (Beta)

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
pkg/util/utilfn/utilfn.go (1)

624-628: Improve the isWinSymlink function implementation.

Consider the following improvements:

  1. Move the WIN_SYMLINK constant outside the function to avoid recreation on each call.
  2. Follow Go naming conventions for the constant.
  3. Add documentation for the function.
+// winSymlink represents the mode bits for a Windows symbolic link
+const winSymlink = os.ModeSymlink
+
+// isWinSymlink checks if the given file mode indicates a Windows symbolic link
 func isWinSymlink(bits os.FileMode) bool {
-    WIN_SYMLINK := os.ModeSymlink
-    return (bits&WIN_SYMLINK > 0)
+    return bits&winSymlink != 0
 }
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ffcab2a and b2cfdc8.

📒 Files selected for processing (1)
  • pkg/util/utilfn/utilfn.go (2 hunks)
🔇 Additional comments (1)
pkg/util/utilfn/utilfn.go (1)

638-638: Verify symlink-to-file handling.

The change correctly identifies Windows symlinks, but we should verify the assumption that all symlinks should be treated as directories. What if a symlink points to a regular file?

✅ Verification successful

Implementation is correct and safe for all symlink types

The isWinSymlink function uses Go's standard os.ModeSymlink check, which correctly identifies symlinks regardless of whether they point to files or directories. The function name is slightly misleading as it works for all symlinks, not just Windows ones, but the implementation is safe and appropriate for skipping mime-type detection on symlinks.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check if there are any symlinks pointing to non-directories in the codebase
# This helps verify our assumption about treating all symlinks as directories

# Look for symlink declarations or creations in tests
rg -l "os\.Symlink|filepath\.Symlink" . --type go

# Look for existing symlinks in the codebase that point to files
find . -type l -exec sh -c 'file -h "{}" | grep -v "directory"' \;

Length of output: 121


Script:

#!/bin/bash
# Get the implementation of isWinSymlink and surrounding context
rg -A 5 -B 5 "func isWinSymlink" .

# Get the full context of DetectMimeType function
ast-grep --pattern 'func DetectMimeType($$$) $$ {
  $$$
}'

# Look for Windows-specific symlink handling patterns
rg -A 3 -B 3 "windows.*symlink|symlink.*windows" . --type go -i

Length of output: 883

@Jalileh
Copy link
Contributor Author

Jalileh commented Jan 10, 2025

Hey for some reason golang fs package uses 0x8000000 which is not the real FILE_ATTRIBUTE value of windows,
unless that value is for cross-compability my changes may screw with linux
image

@esimkowitz
Copy link
Member

I tested this on Linux and it doesn't seem to affect anything so I think we're good there. What tool were you using to make symlinks on Windows? When I use junction, I'm still seeing it show up as a file:
image

@Jalileh
Copy link
Contributor Author

Jalileh commented Jan 16, 2025

I tested this on Linux and it doesn't seem to affect anything so I think we're good there. What tool were you using to make symlinks on Windows? When I use junction, I'm still seeing it show up as a file:
image

mklink

@esimkowitz
Copy link
Member

esimkowitz commented Jan 16, 2025

seems like it should also support junction for completeness, not sure why they operate differently but such is Windows 😅
https://learn.microsoft.com/en-us/sysinternals/downloads/junction

@Jalileh
Copy link
Contributor Author

Jalileh commented Jan 16, 2025

seems like it should also support junction for completeness, not sure why they operate differently but such is Windows 😅
https://learn.microsoft.com/en-us/sysinternals/downloads/junction

Hey give one more commit and I'll see what I can do.

forgot to account for file symlinks and I'm going to have to hardcore the flags for that in think, might as well resolve the junction problem if I can.

@Jalileh
Copy link
Contributor Author

Jalileh commented Jan 16, 2025

image
for some reason golang maps mklink /D and /J as different flags while a file link has the same as /D, why?

@Jalileh
Copy link
Contributor Author

Jalileh commented Jan 16, 2025

image for some reason golang maps mklink /D and /J as different flags while a file link has the same as /D, why?

image
As you can see this makes my whole PR pointless, because of the high level fake flags for windows it breaks file symlinks
To access real window flags id need to make a seperate file for windows im glad to do this if youd like and it would be much easier than rely on this compability interface.

or should I just check the . in the filename to work with the inconsistent symlink lol?

@Jalileh
Copy link
Contributor Author

Jalileh commented Jan 16, 2025

Maybe this isn't necessary? What if /D is uncommon to use? I don't know, but the junction flag wont interfere and if it's the standard then I doubt this bug will be common, i hate cmd.

Added a cheesy workaround to distinguish between files and dir when
softlinks are used.
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b2cfdc8 and 8f06fff.

📒 Files selected for processing (1)
  • pkg/util/utilfn/utilfn.go (2 hunks)
🔇 Additional comments (1)
pkg/util/utilfn/utilfn.go (1)

660-660: LGTM! Directory detection now handles Windows symlinks correctly.

The modification correctly integrates the Windows symlink detection into the mime type detection logic.

Comment on lines 620 to 644
func WinSymlinkDir(path string, bits os.FileMode) bool {
/* for the cursed softlink on windows
the compability fileInfo interface gives us no way to determine its pointing type */
isFileSymlink := func(filepath string) bool {
Length := len(filepath) - 1
maxFileDotExt := 4 // should cover most file extensions
for i := Length; i >= (Length - maxFileDotExt); i-- {
if filepath[i] == '.' {
return true
}
}
return false
}

winSymlink_flags := uint32(bits >> 12)
FLAG_SOFTLINK, FLAG_JUNCTION := uint32(0x8000), uint32(0x80)

if winSymlink_flags == FLAG_SOFTLINK {
return !isFileSymlink(path)
} else if winSymlink_flags == FLAG_JUNCTION {
return true
} else {
return false
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Improve robustness and maintainability of symlink detection.

The implementation has several areas that need improvement:

  1. Magic numbers should be defined as named constants with documentation.
  2. The isFileSymlink function has potential edge cases and could be improved.
  3. Variable naming should follow Go conventions.

Consider applying these improvements:

 // Windows-specific file mode flags for symlinks and junctions
+const (
+    winFlagSoftlink = uint32(0x8000) // FILE_ATTRIBUTE_REPARSE_POINT
+    winFlagJunction = uint32(0x80)   // FILE_ATTRIBUTE_JUNCTION
+)

 func WinSymlinkDir(path string, bits os.FileMode) bool {
-    /* for the cursed softlink on windows
-    the compability fileInfo interface gives us no way to determine its pointing type  */
+    // Windows compatibility layer doesn't expose symlink target type through fileInfo
+    // so we need to check file attributes and extension patterns
     isFileSymlink := func(filepath string) bool {
+        if len(filepath) == 0 {
+            return false
+        }
-        Length := len(filepath) - 1
-        maxFileDotExt := 4 // should cover most file extensions
-        for i := Length; i >= (Length - maxFileDotExt); i-- {
-            if filepath[i] == '.' {
-                return true
-            }
-        }
-        return false
+        return strings.LastIndex(filepath, ".") > strings.LastIndex(filepath, "/")
     }

-    winSymlink_flags := uint32(bits >> 12)
-    FLAG_SOFTLINK, FLAG_JUNCTION := uint32(0x8000), uint32(0x80)
+    flags := uint32(bits >> 12)

-    if winSymlink_flags == FLAG_SOFTLINK {
+    if flags == winFlagSoftlink {
         return !isFileSymlink(path)
-    } else if winSymlink_flags == FLAG_JUNCTION {
+    } else if flags == winFlagJunction {
         return true
     } else {
         return false
     }
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
func WinSymlinkDir(path string, bits os.FileMode) bool {
/* for the cursed softlink on windows
the compability fileInfo interface gives us no way to determine its pointing type */
isFileSymlink := func(filepath string) bool {
Length := len(filepath) - 1
maxFileDotExt := 4 // should cover most file extensions
for i := Length; i >= (Length - maxFileDotExt); i-- {
if filepath[i] == '.' {
return true
}
}
return false
}
winSymlink_flags := uint32(bits >> 12)
FLAG_SOFTLINK, FLAG_JUNCTION := uint32(0x8000), uint32(0x80)
if winSymlink_flags == FLAG_SOFTLINK {
return !isFileSymlink(path)
} else if winSymlink_flags == FLAG_JUNCTION {
return true
} else {
return false
}
}
// Windows-specific file mode flags for symlinks and junctions
const (
winFlagSoftlink = uint32(0x8000) // FILE_ATTRIBUTE_REPARSE_POINT
winFlagJunction = uint32(0x80) // FILE_ATTRIBUTE_JUNCTION
)
func WinSymlinkDir(path string, bits os.FileMode) bool {
// Windows compatibility layer doesn't expose symlink target type through fileInfo
// so we need to check file attributes and extension patterns
isFileSymlink := func(filepath string) bool {
if len(filepath) == 0 {
return false
}
return strings.LastIndex(filepath, ".") > strings.LastIndex(filepath, "/")
}
flags := uint32(bits >> 12)
if flags == winFlagSoftlink {
return !isFileSymlink(path)
} else if flags == winFlagJunction {
return true
} else {
return false
}
}

💡 Codebase verification

Add test coverage for Windows symlink handling

The WinSymlinkDir function lacks test coverage, which is critical for Windows-specific functionality. Please add tests to cover:

  • Windows directory symlinks (mklink /D) with FLAG_SOFTLINK (0x8000)
  • Directory junctions (mklink /J) with FLAG_JUNCTION (0x80)
  • File symlinks with various extensions
  • Edge cases for the file extension detection logic
  • Bit manipulation validation for os.FileMode flags
🔗 Analysis chain

Verify test coverage for Windows symlink handling.

Please ensure that test cases are added to cover:

  1. Directory symlinks created with mklink /D
  2. Directory junctions created with mklink /J
  3. File symlinks
  4. Edge cases with various file extensions

Let's check for existing tests:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for test files that might contain Windows symlink tests
rg -l "mklink|symlink|junction" --type go --glob "*_test.go"

# Search for test functions related to symlinks
ast-grep --pattern 'func Test$_(t *testing.T) {
  $$$
  mklink $$$
  $$$
}'

Length of output: 140


Script:

#!/bin/bash
# Check for test files in the same directory
fd "utilfn_test.go" pkg/util/utilfn/

# Search for WinSymlinkDir usage in test files
rg "WinSymlinkDir" --type go --glob "*_test.go" -C 3

# Look for any symlink-related test code
ast-grep --pattern 'func Test$_(t *testing.T) {
  $$$
  FileMode $$$
  $$$
}'

# Check if the function is used anywhere in the codebase
rg "WinSymlinkDir" -C 3

Length of output: 967

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
pkg/util/utilfn/utilfn.go (1)

628-633: Improve robustness of file extension detection.

The current implementation might have edge cases with paths containing multiple dots. Consider using filepath.Ext() for more reliable extension detection.

 isFileSymlink := func(filepath string) bool {
     if len(filepath) == 0 {
         return false
     }
-    return strings.LastIndex(filepath, ".") > strings.LastIndex(filepath, "/")
+    return filepath.Ext(filepath) != ""
 }
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8f06fff and 69e17c0.

📒 Files selected for processing (1)
  • pkg/util/utilfn/utilfn.go (2 hunks)
🔇 Additional comments (4)
pkg/util/utilfn/utilfn.go (4)

620-623: LGTM! Well-documented Windows file attribute constants.

The constants are correctly typed and accurately represent Windows FILE_ATTRIBUTE flags.


625-644: Add test coverage for Windows symlink handling.

Please add tests to cover:

  1. Directory symlinks created with mklink /D
  2. Directory junctions created with mklink /J
  3. File symlinks with various extensions
  4. Edge cases with empty paths

Would you like me to help create comprehensive test cases for this function?


660-660: LGTM! Clean integration of Windows symlink detection.

The modification elegantly handles Windows symlinks while maintaining the existing behavior for other systems.


635-643: Verify cross-platform compatibility.

While the Windows-specific flags are properly isolated, let's verify that these changes don't affect other platforms.

✅ Verification successful

Cross-platform compatibility verified

The Windows-specific symlink handling is properly isolated in the WinSymlinkDir function and won't affect other platforms. The bit operations using Windows-specific flags are safe as these flag values won't match on other operating systems.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for any existing symlink-related code that might conflict
# Look for other symlink detection code
rg -l "os\.ModeSymlink|symlink|FileMode" --type go

# Check for any platform-specific file mode operations
ast-grep --pattern 'os.FileMode $_ = $_'

Length of output: 172


Script:

#!/bin/bash
# Check symlink-related code implementation in the identified files
rg "os\.ModeSymlink|symlink|FileMode" -A 5 -B 5 pkg/util/utilfn/utilfn.go pkg/wavebase/wavebase.go pkg/wshrpc/wshrpctypes.go

# Look for platform-specific code
rg "windows|linux|darwin" --type go pkg/util/utilfn/utilfn.go

Length of output: 4335

@esimkowitz
Copy link
Member

Just pulled and it looks great to me, thanks for fixing this! Are you ready to merge?

@esimkowitz esimkowitz merged commit bcd0ec8 into wavetermdev:main Jan 18, 2025
7 checks passed
@Jalileh Jalileh deleted the fileMode-fix branch January 28, 2025 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants