Skip to content

Commit

Permalink
Merge pull request #786 from dsyme/docs
Browse files Browse the repository at this point in the history
docs --> docsrc
  • Loading branch information
dsyme authored Jun 28, 2017
2 parents 761bb7e + a69908c commit 74dd8fb
Show file tree
Hide file tree
Showing 539 changed files with 126,403 additions and 33 deletions.
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,6 @@ src/fsharp/fsiAnyCpu/FSIstrings.resx

bin
packages
docs/output
*.bak
temp/
src/fsharp/fsi/FSIStrings.fs
Expand Down Expand Up @@ -191,7 +190,7 @@ artifacts/*.nupkg
*.ldf
.paket/paket.exe
paket-files
docs/tools/FSharp.Formatting.svclog
docsrc/tools/FSharp.Formatting.svclog
src/fsharp/FSharp.Compiler.Service/pplex.fs
src/fsharp/FSharp.Compiler.Service/pppars.fs
src/fsharp/FSharp.Compiler.Service/pppars.fsi
Expand Down
3 changes: 3 additions & 0 deletions RELEASE_NOTES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
#### 13.0.1
* Move docs --> docssrc

#### 13.0.0
* Move FSharp.Compiler.Service.MSBuild.v12.dll to a separate nuget package

Expand Down
18 changes: 3 additions & 15 deletions build.fsx
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ Target "Clean" (fun _ ->
)

Target "CleanDocs" (fun _ ->
CleanDirs ["docs/output"]
CleanDirs ["docs"]
)

Target "Build.NetFx" (fun _ ->
Expand Down Expand Up @@ -208,27 +208,16 @@ Target "PublishNuGet" (fun _ ->
// Generate the documentation

Target "GenerateDocs" (fun _ ->
executeFSIWithArgs "docs/tools" "generate.fsx" ["--define:RELEASE"] [] |> ignore
executeFSIWithArgs "docsrc/tools" "generate.fsx" ["--define:RELEASE"] [] |> ignore
)

Target "GenerateDocsJa" (fun _ ->
executeFSIWithArgs "docs/tools" "generate.ja.fsx" ["--define:RELEASE"] [] |> ignore
executeFSIWithArgs "docsrc/tools" "generate.ja.fsx" ["--define:RELEASE"] [] |> ignore
)

// --------------------------------------------------------------------------------------
// Release Scripts

Target "ReleaseDocs" (fun _ ->
let tempDocsDir = "temp/gh-pages"
if not (System.IO.Directory.Exists tempDocsDir) then
Repository.cloneSingleBranch "" (gitHome + "/" + gitName + ".git") "gh-pages" tempDocsDir

fullclean tempDocsDir
CopyRecursive "docs/output" "temp/gh-pages" true |> printfn "%A"
StageAll tempDocsDir
Commit tempDocsDir (sprintf "Update generated documentation for version %s" buildVersion)
Branches.push "temp/gh-pages"
)

#load "paket-files/fsharp/FAKE/modules/Octokit/Octokit.fsx"
open Octokit
Expand Down Expand Up @@ -424,7 +413,6 @@ Target "All.NetFx" DoNothing
"CleanDocs"
==> "GenerateDocsJa"
==> "GenerateDocs"
==> "ReleaseDocs"
==> "Release"

RunTargetOrDefault "All"
160 changes: 160 additions & 0 deletions docs/caches.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Compiler Services: Notes on the FSharpChecker caches
</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="F# compiler services for creating IDE tools, language extensions and for F# embedding">
<meta name="author" content="Microsoft Corporation, Dave Thomas, Anh-Dung Phan, Tomas Petricek">

<script src="https://code.jquery.com/jquery-1.8.0.js"></script>
<script src="https://code.jquery.com/ui/1.8.23/jquery-ui.js"></script>
<script src="https://netdna.bootstrapcdn.com/twitter-bootstrap/2.2.1/js/bootstrap.min.js"></script>
<link href="https://netdna.bootstrapcdn.com/twitter-bootstrap/2.2.1/css/bootstrap-combined.min.css" rel="stylesheet">

<link type="text/css" rel="stylesheet" href="./content/style.css" />
<link type="text/css" rel="stylesheet" href="./content/fcs.css" />
<script type="text/javascript" src="./content/tips.js"></script>
<!-- HTML5 shim, for IE6-8 support of HTML5 elements -->
<!--[if lt IE 9]>
<script src="https://html5shim.googlecode.com/svn/trunk/html5.js"></script>
<![endif]-->
</head>
<body>
<div class="container">
<div class="masthead">
<ul class="nav nav-pills pull-right">
<li><a href="http://fsharp.org">fsharp.org</a></li>
<li><a href="http://github.com/fsharp/FSharp.Compiler.Service">github page</a></li>
</ul>
<h3 class="muted">F# Compiler Services</h3>
</div>
<hr />
<div class="row">
<div class="span9" id="main">

<h1><a name="Compiler-Services-Notes-on-the-FSharpChecker-caches" class="anchor" href="#Compiler-Services-Notes-on-the-FSharpChecker-caches">Compiler Services: Notes on the FSharpChecker caches</a></h1>
<p>This is a design note on the FSharpChecker component and its caches. See also the notes on the <a href="queue.html">FSharpChecker operations queue</a></p>
<p>Each FSharpChecker object maintains a set of caches. These are</p>
<ul>
<li>
<p><code>scriptClosureCache</code> - an MRU cache of default size <code>projectCacheSize</code> that caches the
computation of GetProjectOptionsFromScript. This computation can be lengthy as it can involve processing the transative closure
of all <code>#load</code> directives, which in turn can mean parsing an unbounded number of script files</p>
</li>
<li>
<p><code>incrementalBuildersCache</code> - an MRU cache of projects where a handle is being kept to their incremental checking state,
of default size <code>projectCacheSize</code> (= 3 unless explicitly set as a parameter).
The "current background project" (see the <a href="queue.html">FSharpChecker operations queue</a>)
will be one of these projects. When analyzing large collections of projects, this cache usually occupies by far the most memory.
Increasing the size of this cache can dramatically decrease incremental computation of project-wide checking, or of checking
individual files within a project, but can very greatly increase memory usage.</p>
</li>
<li><p><code>braceMatchCache</code> - an MRU cache of size <code>braceMatchCacheSize</code> (default = 5) keeping the results of calls to MatchBraces, keyed by filename, source and project options.</p></li>
<li>
<p><code>parseFileInProjectCache</code> - an MRU cache of size <code>parseFileInProjectCacheSize</code> (default = 2) keeping the results of ParseFileInProject,
keyed by filename, source and project options.</p>
</li>
<li>
<p><code>parseAndCheckFileInProjectCache</code> - an MRU cache of size <code>incrementalTypeCheckCacheSize</code> (default = 5) keeping the results of
ParseAndCheckFileInProject, CheckFileInProject and/or CheckFileInProjectIfReady. This is keyed by filename, file source
and project options. The results held in this cache are only returned if they would reflect an accurate parse and check of the
file.</p>
</li>
<li>
<p><code>parseAndCheckFileInProjectCachePossiblyStale</code> - a somewhat peculiar MRU cache of size <code>incrementalTypeCheckCacheSize</code> (default = 5)
keeping the results of ParseAndCheckFileInProject, CheckFileInProject and CheckFileInProjectIfReady,
keyed by filename and project options. This cache is accessed by TryGetRecentTypeCheckResultsForFile. Because the results
are accessed regardless of the content of the file, the checking results returned may be "stale".</p>
</li>
<li><p><code>getToolTipTextCache</code> - an aged lookup cache of strong size <code>getToolTipTextSize</code> (default = 5) computing the results of GetToolTipText.</p></li>
<li>
<p><code>ilModuleReaderCache</code> - an aged lookup of weak references to "readers" for references .NET binaries. Because these
are all weak references, you can generally ignore this cache, since its entries will be automatically collected.
Strong references to binary readers will be kept by other FCS data structures, e.g. any project checkers, symbols or project checking results.</p>
<p>In more detail, the bytes for referenced .NET binaries are read into memory all at once, eagerly. Files are not left
open or memory-mapped when using FSharpChecker (as opposed to FsiEvaluationSession, which loads assemblies using reflection).
The purpose of this cache is mainly to ensure that while setting up compilation, the reads of mscorlib, FSharp.Core and so on
amortize cracking the DLLs.</p>
</li>
<li>
<p><code>frameworkTcImportsCache</code> - an aged lookup of strong size 8 which caches the process of setting up type checking against a set of system
components (e.g. a particular version of mscorlib, FSharp.Core and other system DLLs). These resources are automatically shared between multiple
project checkers which happen to reference the same set of system assemblies.</p>
</li>
</ul>
<p>Profiling the memory used by the various caches can be done by looking for the corresponding static roots in memory profiling traces.</p>
<p>The sizes of some of these caches can be adjusted by giving parameters to FSharpChecker. Unless otherwise noted,
the cache sizes above indicate the "strong" size of the cache, where memory is held regardless of the memory
pressure on the system. Some of the caches can also hold "weak" references which can be collected at will by the GC.</p>
<blockquote>
<p>Note: Because of these caches, uou should generally use one global, shared FSharpChecker for everything in an IDE application.</p>
</blockquote>
<h2><a name="Low-Memory-Condition" class="anchor" href="#Low-Memory-Condition">Low-Memory Condition</a></h2>
<p>Version 1.4.0.8 added a "maximum memory" limit specified by the <code>MaxMemory</code> property on FSharpChecker (in MB). If an FCS project operation
is performed (see <code>CheckMaxMemoryReached</code> in <code>service.fs</code>) and <code>System.GC.GetTotalMemory(false)</code> reports a figure greater than this, then
the strong sizes of all FCS caches are reduced to either 0 or 1. This happens for the remainder of the lifetime of the FSharpChecker object.
In practice this will still make tools like the Visual Studio F# Power Tools usable, but some operations like renaming across multiple
projects may take substantially longer.</p>
<p>By default the maximum memory trigger is disabled, see <code>maxMBDefault</code> in <code>service.fs</code>.</p>
<p>Reducing the FCS strong cache sizes does not guarantee there will be enough memory to continue operations - even holding one project
strongly may exceed a process memory budget. It just means FCS may hold less memory strongly.</p>
<p>If you do not want the maximum memory limit to apply then set MaxMemory to System.Int32.MaxValue.</p>
<h2><a name="Summary" class="anchor" href="#Summary">Summary</a></h2>
<p>In this design note, you learned that the FSharpChecker component keeps a set of caches in order to support common
incremental analysis scenarios reasonably efficiently. They correspond roughly to the original caches and sizes
used by the Visual F# Tools, from which the FSharpChecker component derives.</p>
<p>In long running, highly interactive, multi-project scenarios you should carefully
consider the cache sizes you are using and the tradeoffs involved between incremental multi-project checking and memory usage.</p>


</div>
<div class="span3">
<a href="https://nuget.org/packages/FSharp.Compiler.Service">
<img src="./images/logo.png" style="width:140px;height:140px;margin:10px 0px 0px 35px;border-style:none;" />
</a>
<ul class="nav nav-list" id="menu">
<li class="nav-header">
<a href="./ja/index.html" class="nflag"><img src="./images/ja.png" /></a>
<a href="./index.html" class="nflag nflag2"><img src="./images/en.png" /></a>
F# Compiler Services
</li>
<li><a href="./index.html">Home page</a></li>
<li class="divider"></li>
<li><a href="https://www.nuget.org/packages/FSharp.Compiler.Service">Get Library via NuGet</a></li>
<li><a href="http://github.com/fsharp/FSharp.Compiler.Service">Source Code on GitHub</a></li>
<li><a href="http://github.com/fsharp/FSharp.Compiler.Service/blob/master/LICENSE">License</a></li>
<li><a href="http://github.com/fsharp/FSharp.Compiler.Service/blob/master/RELEASE_NOTES.md">Release Notes</a></li>

<li class="nav-header">Getting started</li>
<li><a href="./index.html">Home page</a></li>
<li><a href="./devnotes.html">Developer notes</a></li>
<li><a href="./fsharp-readme.html">F# compiler readme</a></li>

<li class="nav-header">Available services</li>
<li><a href="./tokenizer.html">F# Language tokenizer</a></li>
<li><a href="./untypedtree.html">Processing untyped AST</a></li>
<li><a href="./editor.html">Using editor (IDE) services</a></li>
<li><a href="./symbols.html">Using resolved symbols</a></li>
<li><a href="./typedtree.html">Using resolved expressions</a></li>
<li><a href="./project.html">Whole-project analysis</a></li>
<li><a href="./interactive.html">Embedding F# interactive</a></li>
<li><a href="./compiler.html">Embedding F# compiler</a></li>
<li><a href="./filesystem.html">Virtualized file system</a></li>

<li class="nav-header">Design Notes</li>
<li><a href="./queue.html">The FSharpChecker operations queue</a></li>
<li><a href="./caches.html">The FSharpChecker caches</a></li>
<li><a href="./corelib.html">Notes on FSharp.Core.dll</a></li>

<li class="nav-header">Documentation</li>
<li><a href="./reference/index.html">API Reference</a>
</li>
</ul>
</div>
</div>
</div>
<a href="http://github.com/fsharp/FSharp.Compiler.Service"><img style="position: absolute; top: 0; right: 0; border: 0;" src="https://s3.amazonaws.com/github/ribbons/forkme_right_gray_6d6d6d.png" alt="Fork me on GitHub"></a>
</body>
</html>
Loading

0 comments on commit 74dd8fb

Please sign in to comment.