FSharp.Formatting error while using FSharp.Data's CsvProvider in ProjectScaffold.

[oscarvarto]

Description

Adding type Stocks = CsvProvider<"./data/MSFT.csv"> to https://github.com/fsprojects/ProjectScaffold/blob/master/src/FSharp.ProjectTemplate/Library.fs causes a FSharp.Formatting Error

Repro steps

Please provide the steps required to reproduce the problem

1.a Use forge new scaffold to create a new Scaffold.
1.b Add nuget FSharp.Data to group Build in paket.dependencies: https://github.com/fsprojects/ProjectScaffold/blob/master/paket.dependencies#L18
1.c Modify https://github.com/fsprojects/ProjectScaffold/blob/master/src/FSharp.ProjectTemplate/Library.fs and call CsvProvider (something like this):

namespace ProcessingData

open FSharp.Data

/// Stock prices example
type Stocks = CsvProvider<"./data/MSFT.csv">

/// Documentation for my library
///
/// ## Example
///
///     let h = Library.hello 1
///     printfn "%d" h
///
module Library = 
  /// Returns 42
  ///
  /// ## Parameters
  ///  - `num` - whatever
  let hello num = 42

1.d Add a data folder inside project folder (in my case [root project folder]/src/ProcessingData) and put http://fsharp.github.io/FSharp.Data/data/MSFT.csv inside it.

2. Use .\build.cmd to run all the targets.

Expected behavior

Build correctly.

Actual behavior

I am getting the following error for GenerateReferenceDocs target:

Starting Target: GenerateReferenceDocs (==> RunTests, GenerateHelp)                                                                                                      
Building documentation (Default), this could take some time, please wait...                                                                                              
C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\packages\build\FAKE\tools\FAKE.exe  --fsiargs -d:FAKE -d:RELEASE -d:REFERENCE "generate.fsx"                      
FSharp.Formatting Information: 0 : FSharp.Formatting Logging setup!                                                                                                      
Yaaf.FSharp.Scripting Information: 0 : Yaaf.FSharp.Scripting Logging setup!                                                                                              
Copying file: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\img\logo-template.pdn                                                                          
Copying file: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\img\logo.png                                                                                   
Copying styles and scripts: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\content\style.css                                                                
Copying styles and scripts: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\content\style.css.bak                                                            
Copying styles and scripts: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\content\style_light.css                                                          
Copying styles and scripts: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\content\tips.js                                                                  
Copying styles and scripts: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\content\img\github-blue.png                                                      
Copying styles and scripts: C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docs\content\img\github.png                                                           
Deleting contents of C:\Users\oscarvarto\gitRepos\dotnetWork\ProcessingData\docsrc\tools\../../docs\reference                                                            
FSharp.Formatting Error: 0 : Could not read comments from entity 'ProcessingData.Stocks': Microsoft.FSharp.Compiler.ErrorLogger+ReportedError: Exception of type 'Microso
ft.FSharp.Compiler.Tast+InternalUndefinedItemRef' was thrown.                                                                                                            
   at Microsoft.FSharp.Compiler.ErrorLogger.ErrorLoggerExtensions.ErrorLogger.Error[b](ErrorLogger x, Exception exn) in C:\Users\robin\dev\FSharp.Compiler.Service\src\fs
harp\ErrorLogger.fs:line 313                                                                                                                                             
   at Microsoft.FSharp.Compiler.Tast.EntityRef.get_Deref() in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\tast.fs:line 2684                                    
   at Microsoft.FSharp.Compiler.Tastops.stripTyEqnsA(TcGlobals g, Boolean canShortcut, TType ty) in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\TastOps.fs:line
 627                                                                                                                                                                     
   at Microsoft.FSharp.Compiler.Tastops.extensionInfoOfTy(TcGlobals g, TType ty) in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\TastOps.fs:line 1464           
   at Microsoft.FSharp.Compiler.Tastops.metadataOfTy(TcGlobals g, TType ty) in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\TastOps.fs:line 1489                
   at Microsoft.FSharp.Compiler.Infos.GetImmediateIntrinsicPropInfosOfType(FSharpOption`1 optFilter, AccessorDomain ad, TcGlobals g, ImportMap amap, range m, TType typ) 
in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\infos.fs:line 3293                                                                                              
   at <StartupCode$FSharp-Compiler-Service>[email protected](IEnumerable`1& next) in C:\Users\robin\dev\FSharp.Compiler.Service\
src\fsharp\vs\Symbols.fs:line 426                                                                                                                                        
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl()                                                                                      
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.System-Collections-IEnumerator-MoveNext()                                                           
   at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source)                                                                                             
   at <StartupCode$FSharp-Compiler-Service>[email protected](Unit unitVar0) in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\vs\
Symbols.fs:line 412                                                                                                                                                      
   at Microsoft.FSharp.Compiler.SourceCodeServices.Impl.protect[a](FSharpFunc`2 f) in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\vs\Symbols.fs:line 27        
   at Microsoft.FSharp.Compiler.SourceCodeServices.FSharpEntity.get_MembersFunctionsAndValues() in C:\Users\robin\dev\FSharp.Compiler.Service\src\fsharp\vs\Symbols.fs:li
ne 411                                                                                                                                                                   
   at [email protected](IEnumerable`1& next) in c:\Tomas\Public\tpetricek\FSharp.Formatting\src\FSharp.MetadataFormat\Main.fs:l
ine 0                                                                                                                                                                    
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl()                                                                                      
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.System-Collections-IEnumerator-MoveNext()                                                           
   at Microsoft.FSharp.Collections.SeqModule.ToList[T](IEnumerable`1 source)                                                                                             
   at [email protected](String cat, IDictionary`2 cmds, Comment comment) in c:\Tomas\Public\tpetricek\FSharp.Formatting\src\FSharp.Metada
taFormat\Main.fs:line 1070                                                                                                                                               
   at FSharp.MetadataFormat.Reader.readCommentsInto[a](FSharpSymbol sym, ReadingContext ctx, String xmlDoc, FSharpFunc`2 f) in c:\Tomas\Public\tpetricek\FSharp.Formattin
g\src\FSharp.MetadataFormat\Main.fs:line 944                                                                                                                             
FSharp.Formatting Information: 0 : Starting razor engine                                                                                                                 
FSharp.Formatting Information: 0 : Generating: index.html                                                                                                                
FSharp.Formatting Information: 0 : Generating modules...                                                                                                                 
FSharp.Formatting Information: 0 : Generating module: processingdata-library                                                                                             
FSharp.Formatting Information: 0 : Finished module: processingdata-library                                                                                               
FSharp.Formatting Information: 0 : Generating types...                                                                                                                   
Finished Target: GenerateReferenceDocs

Known workarounds

None yet :(

Related information

• Operating system
  Windows 10 Pro 64 bits (running on top of Parallels 12 for Mac).

• Branch

• .NET Runtime, CoreCLR or Mono Version
  I have configured framework in Solutions's paket.dependencies to be

framework: >= net461

• Verifying the fsc version:

fsc --help
Microsoft (R) F# Compiler version 4.1

• I have already installed VS 2017 15.3.2

• PATH Environment variables:

PS C:\Users\oscarvarto> $env:path.split(";")
C:\Program Files\ConEmu\ConEmu\Scripts
C:\Program Files\ConEmu
C:\Program Files\ConEmu\ConEmu
C:\Program Files (x86)\Parallels\Parallels Tools\Applications
C:\Windows\system32
C:\Windows
C:\Windows\System32\Wbem
C:\Windows\System32\WindowsPowerShell\v1.0\
C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\110\Tools\Binn\
C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn\
C:\Program Files\Microsoft SQL Server\120\Tools\Binn\
C:\Program Files\Microsoft SQL Server\120\DTS\Binn\
C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn\ManagementStudio\
C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn\
C:\Program Files\Microsoft SQL Server\130\Tools\Binn\
C:\Program Files\dotnet\
C:\Program Files (x86)\Xoreax\IncrediBuild
C:\Program Files\Git\cmd
C:\Program Files (x86)\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\
C:\Program Files (x86)\Microsoft SQL Server\140\Tools\Binn\
C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn\
C:\Program Files (x86)\Microsoft SQL Server\140\Tools\Binn\ManagementStudio\
C:\ProgramData\chocolatey\bin
C:\Program Files (x86)\Microsoft SDKs\F#\4.1\Framework\v4.0
C:\Users\oscarvarto\scoop\shims
C:\Users\oscarvarto\AppData\Local\Microsoft\WindowsApps

• FSHARPINSTALLDIR pointing to C:\Program Files (x86)\Microsoft SDKs\F#\4.1\Framework\v4.0\ (I guess VS 2017 installation added that).

[dsyme]

@oscarvarto Could you add this bug in the FSharp.Formatting repo please? thanks https://github.com/fsprojects/FSharp.Formatting

[oscarvarto]

@dsyme Sure, I added this because of a recommendation I got in the fsharp.slack.com #beginners channel. Thanks for your quick response!

[matthid]

@dsyme Is this a FSharp.Formatting error? It seems the property .FSharpEntity.get_MembersFunctionsAndValues() is throwing. Why?

[dsyme]

@matthid It's possible FSharp.Formatting should be skipping provided types. Or else please provide a repro that involves direct calls to the FCS API - the code for FSharp.Formatting is really hard to follow

[Krzysztof-Cieslak]

Also, FSharp formatting NuGet is really old (over a year old) and is using really outdated version of FCS... so it's hard to know if error is on FCS side, or in F# formatting

[matthid]

And I really hate it when FCS throws cryptic error messages when accessing properties ;).

Repro is retrieving the FSharpEntity of ProcessingData.Stock and accessing the MembersFunctionsAndValues property...

Question is can we detect this before accessing the property and if yes: how?

I can try to get a repro but not anytime soon, but I can tell you it's always straight forward for these kinds of issues ;)

[matthid]

@Krzysztof-Cieslak is correct, please try to use latest alpha package from appveyor :)

[oscarvarto]

I created an issue fsprojects/FSharp.Formatting#450 following @dsyme instructions.

@matthid https://github.com/oscarvarto/ProcessingData might be helpful to reproduce the problem.

@Krzysztof-Cieslak I used F# 4.1 by @pblasucci 's recommendation here fsprojects/ProjectScaffold#309 (comment) (and I think that would make my ionide-vscode dev environment unusable, because of prerequisites here https://github.com/ionide/ionide-vscode-fsharp/#getting-started )

Thanks for your guidance. I will try to use latest alpha package for FSharp.Formatting from appveyor as soon as possible (I still don't know how to do it, but I'll figure it out :)

[dsyme]

Question is can we detect this before accessing the property and if yes: how?

I believe there is an IsProvided property, see https://github.com/Microsoft/visualfsharp/blob/master/src/fsharp/symbols/Symbols.fsi#L215