Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

关于MLKA中的大核和self.LKA3(a_1)*self.X3(a_1)这个地方为什么叫做门控的问题。 #29

Open
sky-1232 opened this issue Dec 17, 2024 · 5 comments
Open

关于MLKA中的大核和self.LKA3(a_1)*self.X3(a_1)这个地方为什么叫做门控的问题。 #29

sky-1232 opened this issue Dec 17, 2024 · 5 comments

Comments

@sky-1232
Copy link

sky-1232 commented Dec 17, 2024 •
edited
Loading

作者打扰您了、这里的LKA中的大核、是指将7×7、9×9这样子的大核卷积分成三个部分来减少参数和计算量、与那种卷积核31*31这种的大核卷积完全不同是吧。还有就是self.LKA3(a_1)*self.X3(a_1)这个地方乘上self.X3(a1)为什么叫做门控机制、而不能叫做注意力机制呢。非常感谢您的代码。
@icandle
Copy link
Owner

icandle commented Dec 17, 2024 •
edited
Loading

这里的大核是类似于31*31的卷积只不过分解成7*7和9*9的卷积大小减小计算量，这里可以参考VAN中的内容，这里是门控不是注意力的原因可以参考#28 中的回答，注意力是对输入的调整可以看作y=a(x)*x而门控粗糙点说可以当作y=f(x)*h(x)

@sky-1232
Copy link
Author

您好、您能给我讲讲门控和注意力的区别在于什么嘛、什么时候用门控什么时候用注意力、我搜索资料、我好像都不太理解。麻烦您了！

@sky-1232
Copy link
Author

或者说注意力机制和门控机制在cv方向来说、本质上都是控制信息的流通。

@sky-1232
Copy link
Author

门控机制更注重特征级别的强化筛选、注意力更注重像素级别的强化筛选？这样理解可以吗

@icandle
Copy link
Owner

icandle commented Dec 19, 2024

不是，上面这两个的公式不一样

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@icandle @sky-1232