Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Servant docs curl #1401

Merged
merged 16 commits into from
Aug 19, 2021
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ doc/_build
doc/venv
doc/tutorial/static/api.js
doc/tutorial/static/jq.js
shell.nix

# nix
result*
Expand Down
3 changes: 1 addition & 2 deletions cabal.project
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@ packages:
servant-client/
servant-client-core/
servant-http-streams/
-- Tests failing with Cabal (TODO: investigate)
-- servant-docs/
servant-docs/
servant-foreign/
servant-server/
doc/tutorial/
Expand Down
16 changes: 16 additions & 0 deletions changelog.d/servant-docs-curl
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
synopsis: Add sample cURL requests to generated documentation
prs: #1401

description: {

Add sample cURL requests to generated documentation.

Those supplying changes to the Request `header` field manually using
lenses will need to add a sample bytestring value.

`headers <>~ ["unicorn"]`

becomes

`headers <>~ [("unicorn", "sample value")]`
}
8 changes: 4 additions & 4 deletions servant-docs/example/greet.hs
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ intro2 = DocIntro "This title is below the last"
-- API specification
type TestApi =
-- GET /hello/:name?capital={true, false} returns a Greet as JSON or PlainText
"hello" :> Capture "name" Text :> QueryParam "capital" Bool :> Get '[JSON, PlainText] Greet
"hello" :> Capture "name" Text :> Header "X-Num-Fairies" Int :> QueryParam "capital" Bool :> Get '[JSON, PlainText] Greet

-- POST /greet with a Greet as JSON in the request body,
-- returns a Greet as JSON
Expand All @@ -93,9 +93,9 @@ testApi = Proxy
extra :: ExtraInfo TestApi
extra =
extraInfo (Proxy :: Proxy ("greet" :> Capture "greetid" Text :> Delete '[JSON] NoContent)) $
defAction & headers <>~ ["unicorns"]
defAction & headers <>~ [("X-Num-Unicorns", "1")]
& notes <>~ [ DocNote "Title" ["This is some text"]
, DocNote "Second secton" ["And some more"]
, DocNote "Second section" ["And some more"]
]

-- Generate the data that lets us have API docs. This
Expand All @@ -109,4 +109,4 @@ docsGreet :: API
docsGreet = docsWith defaultDocOptions [intro1, intro2] extra testApi

main :: IO ()
main = putStrLn $ markdown docsGreet
main = putStrLn $ markdownWith (defRenderingOptions { _renderCurlBasePath = Just "http://localhost:80" }) docsGreet
33 changes: 32 additions & 1 deletion servant-docs/example/greet.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,15 @@ You'll also note that multiple intros are possible.
"Hello, haskeller"
```

### Sample Request:

```bash
curl -XPOST \
-H 'Content-Type: application/json;charset=utf-8 '\
-d "HELLO, HASKELLER" \
http://localhost:80/greet
```

## DELETE /greet/:greetid

### Title
Expand All @@ -67,7 +76,7 @@ And some more

### Headers:

- This endpoint is sensitive to the value of the **unicorns** HTTP header.
- This endpoint is sensitive to the value of the **X-Num-Unicorns** HTTP header.

### Response:

Expand All @@ -85,12 +94,24 @@ And some more

```

### Sample Request:

```bash
curl -XDELETE \
-H 'X-Num-Unicorns: 1' \
http://localhost:80/greet/:greetid
```

## GET /hello/:name

### Captures:

- *name*: name of the person to greet

### Headers:

- This endpoint is sensitive to the value of the **X-Num-Fairies** HTTP header.

### GET Parameters:

- capital
Expand Down Expand Up @@ -120,3 +141,13 @@ And some more
```javascript
"Hello, haskeller"
```

### Sample Request:

```bash
curl -XGET \
-H 'X-Num-Fairies: 1729' \
http://localhost:80/hello/:name
```


70 changes: 53 additions & 17 deletions servant-docs/src/Servant/Docs/Internal.hs
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ import Control.Applicative
import Control.Arrow
(second)
import Control.Lens
(makeLenses, mapped, over, set, traversed, view, (%~), (&),
(.~), (<>~), (^.), (|>))
(makeLenses, mapped, each, over, set, to, toListOf, traversed, view,
_1, (%~), (&), (.~), (<>~), (^.), (|>))
import qualified Data.ByteString.Char8 as BSC
import Data.ByteString.Lazy.Char8
(ByteString)
Expand Down Expand Up @@ -59,6 +59,9 @@ import Data.String.Conversions
import Data.Text
(Text, unpack)
import GHC.Generics
(Generic, Rep, K1(K1), M1(M1), U1(U1), V1,
(:*:)((:*:)), (:+:)(L1, R1))
import qualified GHC.Generics as G
import GHC.TypeLits
import Servant.API
import Servant.API.ContentTypes
Expand Down Expand Up @@ -295,7 +298,7 @@ defResponse = Response
data Action = Action
{ _authInfo :: [DocAuthentication] -- user supplied info
, _captures :: [DocCapture] -- type collected + user supplied info
, _headers :: [Text] -- type collected
, _headers :: [HTTP.Header] -- type collected
, _params :: [DocQueryParam] -- type collected + user supplied info
, _fragment :: Maybe DocFragment -- type collected + user supplied info
, _notes :: [DocNote] -- user supplied
Expand Down Expand Up @@ -356,12 +359,14 @@ data ShowContentTypes = AllContentTypes -- ^ For each example, show each conten
--
-- @since 0.11.1
data RenderingOptions = RenderingOptions
{ _requestExamples :: !ShowContentTypes
{ _requestExamples :: !ShowContentTypes
-- ^ How many content types to display for request body examples?
, _responseExamples :: !ShowContentTypes
, _responseExamples :: !ShowContentTypes
-- ^ How many content types to display for response body examples?
, _notesHeading :: !(Maybe String)
, _notesHeading :: !(Maybe String)
-- ^ Optionally group all 'notes' together under a common heading.
, _renderCurlBasePath :: !(Maybe String)
-- ^ Optionally render example curl requests under a common base path (e.g. `http://localhost:80`).
} deriving (Show)

-- | Default API generation options.
Expand All @@ -373,9 +378,10 @@ data RenderingOptions = RenderingOptions
-- @since 0.11.1
defRenderingOptions :: RenderingOptions
defRenderingOptions = RenderingOptions
{ _requestExamples = AllContentTypes
, _responseExamples = AllContentTypes
, _notesHeading = Nothing
{ _requestExamples = AllContentTypes
, _responseExamples = AllContentTypes
, _notesHeading = Nothing
, _renderCurlBasePath = Nothing
}

-- gimme some lenses
Expand Down Expand Up @@ -412,7 +418,7 @@ docsWithOptions p = docsFor p (defEndpoint, defAction)
-- > extra :: ExtraInfo TestApi
-- > extra =
-- > extraInfo (Proxy :: Proxy ("greet" :> Capture "greetid" Text :> Delete)) $
-- > defAction & headers <>~ ["unicorns"]
-- > defAction & headers <>~ [("X-Num-Unicorns", 1)]
-- > & notes <>~ [ DocNote "Title" ["This is some text"]
-- > , DocNote "Second section" ["And some more"]
-- > ]
Expand Down Expand Up @@ -507,7 +513,7 @@ samples = map ("",)

-- | Default sample Generic-based inputs/outputs.
defaultSamples :: forall a. (Generic a, GToSample (Rep a)) => Proxy a -> [(Text, a)]
defaultSamples _ = second to <$> gtoSamples (Proxy :: Proxy (Rep a))
defaultSamples _ = second G.to <$> gtoSamples (Proxy :: Proxy (Rep a))

-- | @'ToSample'@ for Generics.
--
Expand Down Expand Up @@ -643,7 +649,7 @@ markdown = markdownWith defRenderingOptions
--
-- @since 0.11.1
markdownWith :: RenderingOptions -> API -> String
markdownWith RenderingOptions{..} api = unlines $
markdownWith RenderingOptions{..} api = unlines $
introsStr (api ^. apiIntros)
++ (concatMap (uncurry printEndpoint) . sort . HM.toList $ api ^. apiEndpoints)

Expand All @@ -654,11 +660,12 @@ markdownWith RenderingOptions{..} api = unlines $
notesStr (action ^. notes) ++
authStr (action ^. authInfo) ++
capturesStr (action ^. captures) ++
headersStr (action ^. headers) ++
headersStr (toListOf (headers . each . _1 . to (T.pack . BSC.unpack . CI.original)) action) ++
paramsStr meth (action ^. params) ++
fragmentStr (action ^. fragment) ++
rqbodyStr (action ^. rqtypes) (action ^. rqbody) ++
responseStr (action ^. response) ++
maybe [] (curlStr endpoint (action ^. headers) (action ^. rqbody)) _renderCurlBasePath ++
[]

where str = "## " ++ BSC.unpack meth
Expand Down Expand Up @@ -814,7 +821,6 @@ markdownWith RenderingOptions{..} api = unlines $
("text", "css") -> "css"
(_, _) -> ""


contentStr mime_type body =
"" :
"```" <> markdownForType mime_type :
Expand All @@ -839,6 +845,33 @@ markdownWith RenderingOptions{..} api = unlines $
xs ->
formatBodies _responseExamples xs

curlStr :: Endpoint -> [HTTP.Header] -> [(Text, M.MediaType, ByteString)] -> String -> [String]
curlStr endpoint hdrs reqBodies basePath =
[ "### Sample Request:"
, ""
, "```bash"
, "curl -X" ++ BSC.unpack (endpoint ^. method) ++ " \\"
] <>
maybe [] pure mbMediaTypeStr <>
headersStrs <>
maybe [] pure mbReqBodyStr <>
[ " " ++ basePath ++ showPath (endpoint ^. path)
, "```"
, ""
]

where mbReqBody = listToMaybe reqBodies
mbMediaTypeStr = mkMediaTypeStr <$> mbReqBody
headersStrs = mkHeaderStr <$> hdrs
mbReqBodyStr = mkReqBodyStr <$> mbReqBody
mkMediaTypeStr (_, media_type, _) =
" -H 'Content-Type: " ++ show media_type ++ " '\\"
mkHeaderStr (hdrName, hdrVal) =
" -H '" ++ cs (CI.original hdrName) ++ ": " ++
cs hdrVal ++ "' \\"
mkReqBodyStr (_, _, body) = " -d " ++ cs body ++ " \\"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this correctly escape the request body in case it contains "s?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a great point. Same with the headers. However I think it's up to the implementor to escape those quotes in the ToSample instance. Do you agree?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, well proper escaping will take place for whatever content types are relevant for the request/response bodies (e.g "s are escaped in sample JSON bodies), so I would be inclined to say that the escaping would be fine. But since you're more familiar with this patch than myself for obvious reasons, I was essentially asking for your confirmation. It simply occurred to me that your patch was introducing bash escaping to the mix.

And indeed the same question stands for headers.

(I don't think relying on each sample doing things right is the best way out of this question though.)

Copy link
Contributor Author

@dfithian dfithian Jul 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why the body would be different in this case than in the case where an actual request was being sent, meaning that I expect that quotes would be escaped correctly.

I tested this out, and in fact it doesn't quite do the correct thing; with a (pretty) JSON body, it did something like this:

  -d {
      "foo": "bar"
  } \

So I think we should actually wrap it in single quotes - then we won't have to worry about escaping " anyway. I think I was fooled by the Greet example because it was a JSON string.

Edit: And we don't have to worry about the headers - I checked, and they already are wrapped in single quotes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be good now

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we wrap everything up in single quote, are we sure things will go smoothly in the presence of single quotes inside e.g the request body?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. If there are single quotes inside a JSON request body, they'll show up in the snippet unescaped.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, is there something we can do to guarantee that people will be able to copy/paste any curl commands produced by this code and have it work? This is really the last blocker for me, for this PR, which is otherwise ready to land. Sorry to be a bit of a PITA here, but I'm just thinking about the scenario where in a professional setting people start generating docs with this but other folks in the company would complain about commands not working, etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I could use double quotes and escape double quotes in the block. I'm always wary of doing these things by hand but I'll give it a try

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, done, and greet.md updated.



-- * Instances

-- | The generated docs for @a ':<|>' b@ just appends the docs
Expand Down Expand Up @@ -955,14 +988,17 @@ instance {-# OVERLAPPING #-}
status = fromInteger $ natVal (Proxy :: Proxy status)
p = Proxy :: Proxy a

instance (KnownSymbol sym, HasDocs api)
instance (ToHttpApiData a, ToSample a, KnownSymbol sym, HasDocs api)
=> HasDocs (Header' mods sym a :> api) where
docsFor Proxy (endpoint, action) =
docsFor subApiP (endpoint, action')

where subApiP = Proxy :: Proxy api
action' = over headers (|> headername) action
headername = T.pack $ symbolVal (Proxy :: Proxy sym)
action' = over headers (|> (headerName, headerVal)) action
headerName = CI.mk . cs $ symbolVal (Proxy :: Proxy sym)
headerVal = case toSample (Proxy :: Proxy a) of
Just x -> cs $ toHeader x
Nothing -> "<no header sample provided>"

instance (KnownSymbol sym, ToParam (QueryParam' mods sym a), HasDocs api)
=> HasDocs (QueryParam' mods sym a :> api) where
Expand Down
1 change: 0 additions & 1 deletion servant-docs/test/Servant/DocsSpec.hs
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,6 @@ spec = describe "Servant.Docs" $ do
md `shouldContain` "\"dt1field1\":\"field 1\""
it "contains response samples - dt1field2" $
md `shouldContain` "\"dt1field2\":13"

it "contains request body samples" $
md `shouldContain` "17"

Expand Down