Memory leak in widget.Table due to constantly creating cells to cache.Renderer(impl) while table.Refresh() #4903
Comments
|
Out of interest can you give it half a chance? Wait until the window is visible and also refresh at less than your screens refresh rate? For example a 100ms delay and then a 20ms ticker? |
The testing code just a easy and simple way to reproduce the problem, not a regular scene. Actually I'm using it a very low refresh rate in production environment, such as 2 seconds refresh 4 * 100 cells, but it still take 100mb ram incr every 3 hours. |
|
I think the table code could possibly be improved (looking quickly at the code I don't understand why we need to create a new template cell every refresh) but I don't believe there's a memory leak here. The renderer cache is periodically cleaned out to remove expired renderers for widgets that are not being drawn anymore, so while this will cause memory to go up for a bit, the renderers will eventually be GC'ed. What you may be seeing is Go holding on to more memory than is actually in use by the app too in case it needs to allocate more. Now running refresh every 10 ms will cause a lot of memory to accumulate because IIRC the caches are cleaned every 1 or 2 minutes. |
|
Another thing @ZSA233 if you're updating a table and refreshing for each cell (not sure if you're doing this) - it's better to update all the cells and then refresh the table once. (That is assuming you update all the cells together). In general it's important for performance in Fyne to refresh only when needed, and at the lowest refresh scope too. Basically don't refresh until you want the widget to be redrawn. |
You are right for the cache cleaning code in the driver, so
Showing nodes accounting for 233.01MB, 91.87% of 253.63MB total
Dropped 121 nodes (cum <= 1.27MB)
Showing top 20 nodes out of 143
flat flat% sum% cum cum%
35.51MB 14.00% 14.00% 52.51MB 20.70% fyne.io/fyne/v2/widget.(*RichText).cachedSegmentVisual
28.72MB 11.32% 25.32% 163.59MB 64.50% fyne.io/fyne/v2/internal/cache.Renderer
26.61MB 10.49% 35.81% 26.61MB 10.49% github.com/go-text/typesetting/opentype/loader.(*Loader).findTableBuffer
26.01MB 10.25% 46.07% 28.01MB 11.04% fyne.io/fyne/v2/widget.NewLabelWithStyle
18MB 7.10% 53.17% 18MB 7.10% fyne.io/fyne/v2/widget.NewRichText (inline)
16.50MB 6.51% 59.67% 16.50MB 6.51% fyne.io/fyne/v2/canvas.NewText (inline)
12.51MB 4.93% 64.60% 12.51MB 4.93% github.com/go-text/typesetting/opentype/tables.(*SimpleGlyph).parsePoints
12MB 4.73% 69.34% 12MB 4.73% fyne.io/fyne/v2/canvas.NewRectangle (inline)
11MB 4.34% 73.67% 11MB 4.34% runtime.malg
10MB 3.94% 77.62% 28MB 11.04% fyne.io/fyne/v2/widget.NewRichTextWithText
5.50MB 2.17% 79.79% 19.01MB 7.49% github.com/go-text/typesetting/opentype/tables.(*Glyph).parseData
4.50MB 1.77% 81.56% 4.50MB 1.77% time.NewTicker
4.50MB 1.77% 83.33% 4.50MB 1.77% fyne.io/fyne/v2/internal/cache.(*expiringCache).setAlive
4.15MB 1.64% 84.97% 4.15MB 1.64% github.com/riobard/go-bloom.New
3.50MB 1.38% 86.35% 15.21MB 6.00% fyne.io/fyne/v2/widget.(*RichText).updateRowBounds.func1
3.50MB 1.38% 87.73% 3.50MB 1.38% fyne.io/fyne/v2/internal/widget.NewSimpleRenderer (inline)
3MB 1.18% 88.91% 76.01MB 29.97% fyne.io/fyne/v2/widget.(*RichText).CreateRenderer
3MB 1.18% 90.10% 3MB 1.18% fyne.io/fyne/v2/widget.(*BaseWidget).ExtendBaseWidget
2.50MB 0.99% 91.08% 55.51MB 21.89% fyne.io/fyne/v2/widget.(*textRenderer).Refresh
2MB 0.79% 91.87% 2MB 0.79% fyne.io/fyne/v2/widget.splitLines
|
|
This could be a duplicate of text cache issues we are working on, if the text in the cells is changing on each refresh. |
|
Could be, though I still don't think it's an actual memory leak, as the text texture cache is cleaned out on a schedule too. It's an issue of more memory being consumed than needed though since each unique line of text (as opposed to glyph) shown is a unique entry in the texture cache. |
|
It shouldn't grow like the OP is reporting though surely - over hours of usage? |
Yeap,
testing result
PS C:\Users\ZSA> go tool pprof http://127.0.0.1:9996/debug/pprof/heap
Fetching profile over HTTP from http://127.0.0.1:9996/debug/pprof/heap
Saved profile in C:\Users\ZSA\pprof\pprof.___6go_build_test.exe.alloc_objects.alloc_space.inuse_objects.inuse_space.001.pb.gz
File: ___6go_build_test.exe
Build ID: C:\Users\ZSA\AppData\Local\JetBrains\GoLand2024.1\tmp\GoLand\___6go_build_test.exe2024-06-07 00:59:15.714434 +0800 CST
Type: inuse_space
Time: Jun 7, 2024 at 1:03am (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top20
Showing nodes accounting for 23784.30kB, 100% of 23784.30kB total
Showing top 20 nodes out of 131
flat flat% sum% cum cum%
9605.13kB 40.38% 40.38% 9605.13kB 40.38% github.com/go-text/typesetting/opentype/loader.(*Loader).findTableBuffer
2560.70kB 10.77% 51.15% 2560.70kB 10.77% fyne.io/fyne/v2/widget.(*RichText).cachedSegmentVisual
2272.37kB 9.55% 60.70% 2272.37kB 9.55% fyne.io/fyne/v2/internal/cache.SetFontMetrics
1056.33kB 4.44% 65.15% 1056.33kB 4.44% github.com/go-text/typesetting/opentype/tables.ParseBaseArray
1024.20kB 4.31% 69.45% 1024.20kB 4.31% fyne.io/fyne/v2/widget.NewLabelWithStyle
1024.19kB 4.31% 73.76% 8044.71kB 33.82% fyne.io/fyne/v2/widget.NewRichText (inline)
1024.06kB 4.31% 78.06% 2048.21kB 8.61% github.com/go-text/typesetting/opentype/tables.(*Glyph).parseData
596.16kB 2.51% 80.57% 596.16kB 2.51% github.com/go-text/typesetting/opentype/api/font/cff.parseIndexContent
522.70kB 2.20% 82.77% 522.70kB 2.20% github.com/go-text/typesetting/unicodedata.map.init.0
513.50kB 2.16% 84.93% 513.50kB 2.16% image.NewRGBA
512.44kB 2.15% 87.08% 512.44kB 2.15% github.com/go-text/typesetting/harfbuzz.newShaperOpentype
512.17kB 2.15% 89.24% 512.17kB 2.15% github.com/go-text/typesetting/opentype/api/font.(*Face).getPointsForGlyph
512.12kB 2.15% 91.39% 512.12kB 2.15% github.com/go-text/typesetting/harfbuzz.(*otMap).addLookups
512.10kB 2.15% 93.54% 512.10kB 2.15% github.com/go-text/typesetting/opentype/tables.(*SimpleGlyph).parsePoints
512.05kB 2.15% 95.69% 512.05kB 2.15% regexp/syntax.(*parser).newRegexp
512.05kB 2.15% 97.85% 512.05kB 2.15% github.com/go-text/typesetting/opentype/tables.(*CompositeGlyph).parseGlyphs 512.02kB 2.15% 100% 512.02kB 2.15% fyne.io/fyne/v2/internal/cache.SetCanvasForObject
0 0% 100% 15578.21kB 65.50% fyne.io/fyne/v2.MeasureText
0 0% 100% 16890.72kB 71.02% fyne.io/fyne/v2/internal/cache.Renderer
0 0% 100% 2562.25kB 10.77% fyne.io/fyne/v2/internal/driver.WalkVisibleObjectTree
(pprof)
PS C:\Users\ZSA> tasklist /FI "PID eq 11876"
映像名称 PID 会话名 会话# 内存使用
========================= ======== ================ =========== ============
___6go_build_test.exe 11876 Console 1 155,352 K
PS C:\Users\ZSA> go tool pprof http://127.0.0.1:9996/debug/pprof/heap
Fetching profile over HTTP from http://127.0.0.1:9996/debug/pprof/heap
Saved profile in C:\Users\ZSA\pprof\pprof.___6go_build_test.exe.alloc_objects.alloc_space.inuse_objects.inuse_space.005.pb.gz
File: ___6go_build_test.exe
Build ID: C:\Users\ZSA\AppData\Local\JetBrains\GoLand2024.1\tmp\GoLand\___6go_build_test.exe2024-06-07 00:59:15.714434 +0800 CST
Type: inuse_space
Time: Jun 7, 2024 at 11:56am (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top20
Showing nodes accounting for 440.77MB, 96.11% of 458.63MB total
Dropped 86 nodes (cum <= 2.29MB)
Showing top 20 nodes out of 82
flat flat% sum% cum cum%
135.54MB 29.55% 29.55% 173.04MB 37.73% fyne.io/fyne/v2/widget.(*RichText).cachedSegmentVisual
69.51MB 15.16% 44.71% 69.51MB 15.16% fyne.io/fyne/v2/widget.NewLabelWithStyle (inline)
55.01MB 11.99% 56.70% 61.87MB 13.49% fyne.io/fyne/v2/widget.NewRichText (inline)
36MB 7.85% 64.55% 36MB 7.85% fyne.io/fyne/v2/canvas.NewText
32MB 6.98% 71.53% 93.87MB 20.47% fyne.io/fyne/v2/widget.NewRichTextWithText
25MB 5.45% 76.98% 25MB 5.45% fyne.io/fyne/v2/canvas.NewRectangle (inline)
22.50MB 4.91% 81.89% 43.71MB 9.53% fyne.io/fyne/v2/widget.(*RichText).updateRowBounds.func1
21.81MB 4.76% 86.65% 370.87MB 80.86% fyne.io/fyne/v2/internal/cache.Renderer
10.50MB 2.29% 88.94% 250.69MB 54.66% fyne.io/fyne/v2/widget.(*RichText).CreateRenderer
9.38MB 2.05% 90.98% 9.38MB 2.05% github.com/go-text/typesetting/opentype/loader.(*Loader).findTableBuffer
6MB 1.31% 92.29% 6MB 1.31% fyne.io/fyne/v2/widget.splitLines
5.50MB 1.20% 93.49% 181.54MB 39.58% fyne.io/fyne/v2/widget.(*textRenderer).Refresh
4.50MB 0.98% 94.47% 4.50MB 0.98% fyne.io/fyne/v2/internal/cache.(*expiringCache).setAlive
4MB 0.87% 95.34% 4MB 0.87% fyne.io/fyne/v2/theme.darkPaletColorNamed
3MB 0.65% 96.00% 4.50MB 0.98% github.com/go-text/typesetting/harfbuzz.NewFont
0.50MB 0.11% 96.11% 8.02MB 1.75% fyne.io/fyne/v2/internal/driver/glfw.(*gLDriver).startDrawThread.func1
0 0% 96.11% 15.21MB 3.32% fyne.io/fyne/v2.MeasureText
0 0% 96.11% 13.52MB 2.95% fyne.io/fyne/v2/internal/driver.WalkVisibleObjectTree
0 0% 96.11% 13.52MB 2.95% fyne.io/fyne/v2/internal/driver.walkObjectTree
0 0% 96.11% 13.52MB 2.95% fyne.io/fyne/v2/internal/driver.walkObjectTree.func1 (inline)
(pprof)
PS C:\Users\ZSA> tasklist /FI "PID eq 11876"
映像名称 PID 会话名 会话# 内存使用
========================= ======== ================ =========== ============
___6go_build_test.exe 11876 Console 1 1,009,812 K
testing codemain.gomain.go
package main
import (
"fmt"
"fyne.io/fyne/v2"
"fyne.io/fyne/v2/app"
"strconv"
"time"
)
var rows = [][]string{}
func main() {
myApp := app.New()
myWindow := myApp.NewWindow("test")
myWindow.Resize(fyne.NewSize(1000, 500))
headers := []string{}
for i := 0; i < 100; i++ {
rows = append(rows, make([]string, 100))
headers = append(headers, "#"+strconv.Itoa(i))
for j := 0; j < 100; j++ {
rows[i][j] = fmt.Sprintf("%d:%d", i, j)
}
}
table := NewTableEx(headers)
myWindow.SetContent(table)
go runPprof()
go refreshLoop(table)
myWindow.ShowAndRun()
}
func refreshLoop(table *TableEx) {
tick := time.NewTicker(time.Millisecond * 500)
defer tick.Stop()
for _ = range tick.C {
for i := 0; i < 100; i++ {
for j := 0; j < 100; j++ {
rows[i][j] = fmt.Sprintf("%d:%d", time.Now().Second()+i, time.Now().Second()+j)
}
}
table.SetMatrix(rows)
}
}table.gopackage main
import (
"fyne.io/fyne/v2"
"fyne.io/fyne/v2/widget"
"strconv"
"sync/atomic"
)
type TableEx struct {
*widget.Table
header atomic.Pointer[[]string]
rows atomic.Pointer[[][]string]
}
func (w *TableEx) Resize(size fyne.Size) {
w.Table.Resize(size)
}
func (w *TableEx) SetMatrix(ss [][]string) {
w.rows.Store(&ss)
w.Table.Refresh()
}
func NewTableEx(header []string) *TableEx {
table := widget.NewTableWithHeaders(nil, nil, nil)
table.ShowHeaderRow = true
table.ShowHeaderColumn = true
var hlp *TableEx
hlp = &TableEx{
header: atomic.Pointer[[]string]{},
rows: atomic.Pointer[[][]string]{},
Table: table,
}
hlp.header.Store(&header)
hlp.rows.Store(&[][]string{})
table.Length = func() (rows int, cols int) {
return len(*hlp.rows.Load()), len(*hlp.header.Load())
}
table.CreateCell = func() fyne.CanvasObject {
return widget.NewLabelWithStyle("", fyne.TextAlignCenter, fyne.TextStyle{})
}
table.CreateHeader = func() fyne.CanvasObject {
return widget.NewLabelWithStyle("", fyne.TextAlignCenter, fyne.TextStyle{
Bold: true,
})
}
table.UpdateCell = func(id widget.TableCellID, template fyne.CanvasObject) {
l := template.(*widget.Label)
var text string
rows := *hlp.rows.Load()
if id.Row < len(rows) && len(rows[id.Row]) > id.Col {
text = rows[id.Row][id.Col]
}
l.SetText(text)
}
table.UpdateHeader = func(id widget.TableCellID, template fyne.CanvasObject) {
l := template.(*widget.Label)
if id.Row < 0 {
h := *hlp.header.Load()
if id.Col >= len(h) {
} else {
l.SetText(h[id.Col])
}
} else if id.Col < 0 {
l.SetText(strconv.Itoa(id.Row + 1))
if l.Alignment != fyne.TextAlignTrailing {
l.Alignment = fyne.TextAlignTrailing
}
} else {
return
}
}
return hlp
}debug.gopackage main
import (
"net/http"
"net/http/pprof"
)
func runPprof() {
//http.HandleFunc("/debug/pprof/", pprof.Index)
//http.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
//http.HandleFunc("/debug/pprof/profile", pprof.Profile)
//http.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
//http.HandleFunc("/debug/pprof/trace", pprof.Trace)
http.HandleFunc("/debug/pprof/allocs", pprof.Handler("allocs").ServeHTTP)
http.HandleFunc("/debug/pprof/block", pprof.Handler("block").ServeHTTP)
http.HandleFunc("/debug/pprof/goroutine", pprof.Handler("goroutine").ServeHTTP)
http.HandleFunc("/debug/pprof/heap", pprof.Handler("heap").ServeHTTP)
http.HandleFunc("/debug/pprof/mutex", pprof.Handler("mutex").ServeHTTP)
http.HandleFunc("/debug/pprof/threadcreate", pprof.Handler("threadcreate").ServeHTTP)
http.ListenAndServe("0.0.0.0:9996", nil)
} |
|
|
If there is indeed a leak, I imagine it must be in GPU textures, as I'm less familiar with how that part of the codebase works and I'm pretty sure there's no leak on the widget/renderer side of things and awhile back did a bit of a codebase audit looking for leaks, fixed a few, but didn't find any in this area. Or else it's Go holding on to memory because the OS isn't under memory pressure and asking for it back. AFAIK, the Go runtime marks freed memory as "unneeded" on most OSes but retains it, or at least a chunk of it, unless the OS asks for it back, to speed future allocations. |
I’m no pprof magician but it feels like the built-in profiling in Go should report actual me memory usage and not how much it is holding on to? My best bet is that this is an actual leak somewhere. |
|
From what I can see type TableEx struct {
*widget.Tableshould be: type TableEx struct {
widget.Tableand there is no call to If the code will not produce the same leak with the regular |
|
I'm having a similar problem, once per second refresh the table and the memory will grow fast, about 1M a second. after some time passes, a gc will be triggered and the memory will be reduced from 200M to 50M. then it will grow slowly again. Much slower than the initial growth. first time: the other time: |
|
We cache all the graphical textures, which includes text rendered. When nothing has been shown the cache will grow fast. Then we clean out unused textures (approx each minute) and re-used items will stay in memory so growth later on will indeed be slower. I'm not sure that this indicates a memory leak as per the original post. |
The cache is supposed to be cleaned periodically, but the glfw driver does not do this as the mobile driver does. This commit adds similar logic to the glfw driver as exists in the mobile driver, to periodically clean the cache on paint events. Fixes fyne-io#4903
The cache is supposed to be cleaned periodically, but the glfw driver does not do this as the mobile driver does. This commit adds similar logic to the glfw driver as exists in the mobile driver, to periodically clean the cache on paint events. Fixes fyne-io#4903
|
@andydotxyz In the current glfw driver, the cache never gets cleaned. I looked at the mobile driver and did more or less what it was doing and was able to fix my issue, and also fix the issue in the testcase code in #4903 (comment) My PR: #5112 @ZSA233 are you able to see if this fixes your issue? As a side note, I noticed two things:
|
dweymouth
added
bug
|
Marking as blocker because this does seem to be a true memory leak (widget renderers are never destroyed), and should be a lot easier to solve with the new threading model. Whatever implementation must also solve #5131 |
Checklist
Describe the bug
widget.Table will CreateCell every time when table.Refresh() called.
detail below
How to reproduceHow to reproduce
Create Cell: xxxxxprinting constantly.widget.Labelintorenderers. Consequently, no gc reach.Screenshots
No response
Example code
see
how to reproduce.Fyne version
2.4.5
Go compiler version
1.22.3
Operating system and version
macOS 13.1
Additional Information
No response
The text was updated successfully, but these errors were encountered: