feat: document architecture, q&a, improve README and other docs

daniel-orlov · Jul 14, 2023 · 0a965c6 · 0a965c6
1 parent 641d1ac
commit 0a965c6
Show file tree

Hide file tree

Showing 5 changed files with 209 additions and 10 deletions.
diff --git a/README.md b/README.md
@@ -1,5 +1,13 @@
 # quotes-server
 
+Quotes server is the backend part of the test assignment solution for DDOS-protected TCP server.
+
+For more details on the technical requirements, see [docs/000_Requirements.md](docs/000_Requirements.md).
+
+Please also
+consult [docs/003_Potential_questions_and_answers_to_them.md](docs/003_Potential_questions_and_answers_to_them.md) for
+additional information on the project.
+
 ## Requirements
 
 + [Go 1.19+](https://go.dev/dl/) installed (to run tests, start server or client without Docker)
@@ -10,6 +18,39 @@
 
 ## Getting started
 
+### Environment variables
+
+To avoid overcomplicating testing the solution of the project, I used envconfig with some reasonable defaults.
+You can find them in the [config/config.go](config/config.go) file for server and in
+the [pkg/client/config.go](pkg/client/config.go) file for client.
+
+If you would like to set them manually, you can do it via environment variables:
+
+### Server
+
+| Name                 | Description                               | Default Value | Possible Values                 |
+|----------------------|-------------------------------------------|---------------|---------------------------------|
+| LOG_LEVEL            | Log level to use                          | debug         | debug, info, warn, error, fatal |
+| LOG_FORMAT           | Log format to use                         | console       | console, json                   |
+| SERVER_PORT          | Port to listen on                         | 8080          | any port you find reasonable    |
+| RATELIMITER_RATE     | Rate at which requests are allowed        | second        | second, minute                  |
+| RATELIMITER_LIMIT    | Maximum number of requests allowed        | 5             |                                 |
+| RATELIMITER_KEY      | Key to use for the ratelimiter            | client_ip     | client_ip                       |
+| CHALLENGE_DIFFICULTY | Difficulty of the proof of work challenge | 20            | 1 to 30 (recommended)           |
+| SALT_LENGTH          | Length of the salt                        | 8             |                                 |
+
+### Client
+
+| Name                    | Description                               | Default Value     | Possible Values                                                  |
+|-------------------------|-------------------------------------------|-------------------|------------------------------------------------------------------|
+| LOG_LEVEL               | Log level to use                          | debug             | debug, info, warn, error, fatal                                  |
+| LOG_FORMAT              | Log format to use                         | console           | console, json                                                    |
+| SERVER_HOST             | Host of the server to connect to          | localhost         | wherever server is hosted                                        |
+| SERVER_PORT             | Port of the server to connect to          | 8080              | whichever port server is listebing on                            |
+| REQUEST_PATH            | Path of the request to send to the server | /v1/quotes/random | whichever endpoint you want to hit on server                     |
+| REQUEST_RATE_PER_SECOND | Number of requests per second to send     | 100               |                                                                  |
+| REQUEST_COUNT           | Number of requests to send to the server  | 0                 | 0 means "run indefinetily", any positive number would limit that |
+
 ### Start server and client via docker-compose:
 
 ```
@@ -36,6 +77,10 @@ make test
 
 ## Project structure
 
+The structure of the project is inspired by Standard Go Project Layout.
+For more information on the architectural thinking behind this structure,
+see [docs/002_Project_architecture.md](docs/002_Project_architecture.md).
+
 > / **quotes-server**
 >
 > > / **.github**
@@ -85,4 +130,9 @@ make test
 
 ## More information
 
-For more information, please, refer to /docs directory.
+For more information, please, refer to /docs directory.
+
+## Infrastructure
+
+The project is hosted on GitHub and uses GitHub Actions for CI/CD pipelines.
+For actual infrastructure, see infrastructure repository - [link, WIP](https://github.com/daniel-orlov/quotes-infra)
diff --git a/docs/000_Requirements.md b/docs/000_Requirements.md
@@ -2,31 +2,31 @@
 
 ## Design and implement “Word of Wisdom” tcp server.
 
-- [ ] TCP server should be protected from DDOS attacks with the Prof of
+- [x] TCP server should be protected from DDOS attacks with the Prof of
   Work (https://en.wikipedia.org/wiki/Proof_of_work), the challenge-response protocol should be used.
 
-- [ ] The choice of the POW algorithm should be explained.
+- [x] The choice of the POW algorithm should be explained.
 
-- [ ] After Proof Of Work verification, server should send one of the quotes from “word of wisdom” book or any other
+- [x] After Proof Of Work verification, server should send one of the quotes from “word of wisdom” book or any other
   collection of the quotes.
 
-- [ ] Docker file should be provided both for the server and for the client that solves the POW Challenge.
+- [x] Docker file should be provided both for the server and for the client that solves the POW Challenge.
 
-- [ ] Code should be covered with unit tests.
+- [x] Code should be covered with unit tests.
 
-- [ ] Code should be fully covered in comments (methods, variables, tests, etc.).
+- [x] Code should be fully covered in comments (methods, variables, tests, etc.).
 
 - [x] Code should be published on GitHub.
 
-- [ ] Readme file should be provided with instructions on how to run the server and the client.
+- [x] Readme file should be provided with instructions on how to run the server and the client.
 
 ### Additional requirements (as derrived from the answers to the questions)
 
-- [ ] PoW algorithm should be balance between computational complexity and memory requirements.
+- [x] PoW algorithm should be balance between computational complexity and memory requirements.
 
 - [x] No constraints on the choice of the storage for the quotes.
 
-- [ ] Additional layers of protection against DDoS attacks could be added, such as rate limiting or IP blocking.
+- [x] Additional layers of protection against DDoS attacks could be added, such as rate limiting or IP blocking.
 
 - [x] Performance requirements for the server are not defined.
 

diff --git a/docs/001_PoW_algorithm_choice.md b/docs/001_PoW_algorithm_choice.md
@@ -37,5 +37,6 @@ community support make it a suitable choice. While Scrypt offers certain advanta
 of this project lean towards Hashcash as the optimal solution.
 
 ## References:
+
 - [Hashcash](http://hashcash.org/papers/hashcash.pdf)
 - [Scrypt](https://www.tarsnap.com/scrypt/scrypt.pdf)
diff --git a/docs/002_Project_architecture.md b/docs/002_Project_architecture.md
@@ -0,0 +1,36 @@
+# Project architecture
+
+## 1. Introduction
+
+For this project I chose Go language, because it's simple, fast and has good concurrency support and developed
+ecosystem.
+For the architecture of the project I chose hexagonal-inspired n-tier architecture.
+Let me explain why and what it means in practice.
+
+## 2. Architecture
+
+![Screenshot 2023-07-14 at 09.32.55.png](..%2F..%2F..%2F..%2F..%2F..%2F..%2F..%2Fvar%2Ffolders%2Fgq%2F20h5x7y916g1x5f6kgmd34s40000gn%2FT%2FTemporaryItems%2FNSIRD_screencaptureui_gQUeXi%2FScreenshot%202023-07-14%20at%2009.32.55.png)
+
+This architecture is based on the idea of separation of concerns and single responsibility principle.
+There are 3 main layers (or tiers):
+
++ Application layer aka service layer - contains business logic of the application
++ Transport layer - contains logic of communication with external systems
++ Storage layer - contains logic of communication with storage systems
+
+This layered leverages the fact that each layer has its own responsibility
+and could be changed without affecting other layers.
+Layers are kept agnostic to each other by using interfaces.
+That allows to easily change implementation of some layer without affecting other layers, for example,
+If I wanted to introduce gRPC server instead of TCP server, I could easily do it by changing only transport layer.
+Similarly, if I wanted to change storage from in-memory to Redis, I could do it by changing only storage layer.
+This solid foundation provides a lot of extensibility and flexibility for the project.
+This also provided decent testability for the project, because I could easily mock any layer and test it in isolation.
+
+As any n-tier architecture, this architecture has downside — I had to write more code, but in this case, there were no
+unnecessary proxing or forced layer additions, so I think that it was worth it.
+
+## 3. Read more
+
+- hexagonal architecture: [link](https://github.com/golang-standards/project-layout)
+- project structure: [link](https://github.com/golang-standards/project-layout)
diff --git a/docs/003_Potential_questions_and_answers_to_them.md b/docs/003_Potential_questions_and_answers_to_them.md
@@ -0,0 +1,112 @@
+# Potential question and answers to them
+
+Since this project is a solution for a technical challenge that is supposed to be presented without any additional
+explanations, I decided to write down some potential questions and answers to them.
+
+The questions are listed in no particular order.
+
+## Why did you choose this particular PoW algorithm?
+
+Short answer: hashcash has a proven track record as an anti-spam mechanism, it is a good fit for the task, and it is
+simple to implement.
+Long answer: please see [docs/001_PoW_algorithm_choice.md](001_PoW_algorithm_choice.md).
+
+## Can you explain how your solution is helping in preventing DDOS attacks?
+
+Sure!
+I based my solution on the combination of two mechanisms: PoW and rate limiting.
+Both are common and proven mechanisms for preventing DDOS attacks, and they complement each other nicely.
+Frankly, rate limiting might have been enough on its own, but since PoW is a requirement, it just made the server more
+secure.
+
+## Why have you used self-written mocks instead of generated ones?
+
+While I have positive experience with generated mocks (like mockgen and mockery),
+in recent years I have been transitioning towards self-written mocks for the following reasons:
+
+- Generated mocks are not very readable and require additional time to understand them.
+- Generated mocks have required me to pass controller, write all the expectations for them to know what to return,
+  which polluted my tests with unnecessary code and made them less readable.
+- Self-written mocks are more readable and require less code to write, even if we take into account the fact that
+  they need to be maintained, and they need tests for themselves.
+
+All things considered, I think that self-written mocks are more readable and maintainable, and I prefer them over the
+generated ones.
+
+## Why have you not used Redis/Memcached/Postgres for storing quotes/challenges?
+
+Instead of using a database, I decided to use in-memory storage for quotes and challenges, but due to the fact that I
+have used interfaces for storage, it is possible to easily switch to any other storage implementation, including Redis,
+Memcached, Postgres, etc. This allowed me to focus on the core functionality of the project and not waste time on the
+database, which wouldn't be a big challenge to add anyway.
+
+## Do you really write that many comments in your code, or is it just for the sake of this project?
+
+The latter.
+I think that comments should be used sparingly and only when they add value,
+but the requirements for this project state that "code should be fully covered in comments", so I did it.
+I guess that the idea behind this requirement is to see how I write comments and how I structure my code,
+as well as be able to see my reasoning and logic, because I will not be able to explain it in person in a demo.
+
+## Why is this project written using Go 1.19, while the latest version is 1.20 at the moment?
+
+Staying one version behind the latest Go release for a project offers benefits such as stability, compatibility with
+third-party dependencies, ecosystem support, mature tooling, and reduced risk of encountering undiscovered bugs or
+compatibility issues.
+Even though, it may result in missing out on the latest language features and performance improvements,
+I think that in most cases,
+unless you really need a new feature or there's a critical bug fix or vulnerability,
+staying one version behind is a good tactic.
+
+## Why have you used X package/library?
+
+Let's go over the list of dependencies and I will explain why I have chosen them.
+github.com/JGLTechnologies/gin-rate-limit v1.5.4: Provides rate limiting middleware for Gin framework, allowing you to
+control and limit the number of requests to your application.
+
+- gin-goinc/gin: I like that it's fast and flexible, with a clean and intuitive API design.
+  It also is a very popular framework, which means that it has a big community and a lot of third-party libraries.
+- JGLTechnologies/gin-rate-limit: speaking of third-party libraries, I have chosen this one because it is the most
+  popular
+  rate limiting middleware for Gin framework, and it is actively maintained.
+- kelseyhightower/envconfig:
+  Simplifies the process
+  of reading configuration data from environment variables by automatically mapping them to Go structs (and has
+  defaults).
+  Lovely, simple, and easy to use.
+- oklog/ulid/v2: Implements Universally Unique Lexicographically Sortable Identifiers (ULID),
+  which are highly efficient, and URL-safe alternatives to traditional UUIDs.
+  Would be a good fit for most identifiers, but I have used it only for quote IDs.
+- stretchr/testify: A popular and comprehensive testing toolkit for Go, providing powerful assertion functions and
+  utilities to simplify writing tests and improve test coverage.
+- ybbus/httpretry: Offers an easy way to perform retry logic for HTTP requests,
+  allowing you to handle transient failures and improve the reliability of your applications.
+  It is also very easy to use, and it complies with http.Client interface, so it can be used with almost any HTTP
+  client.
+- go.uber.org/zap: A fast, structured, and highly efficient logging library for Go, designed for high-performance
+  applications with minimal memory allocations and low overhead. It is also very easy to use and has a lot of features.
+
+## What would you do if you had more time?
+
+My list of things to do would look like this (grouped by priority):
+
+It might be that I have already added that by the time you are reading this:
+
++ Add more tests for the client (I think that it is not enough), especially integration tests.
++ Finish infrastructure setup (in another repo, since it is not a part of this
+  project - [repo link](https://github.com/daniel-orlov/quotes-infra)
+
+Will be adding in the future:
+
++ Improve error responses from server (currently they are not very informative and ad hoc defined, I would like client
+  to be able to rely on the same error structure for all types of errors and for server to be able to manipulate and
+  centrally control error type definition)
++ Add telemetry for more observability (currently there are logs, which are good, but not enough)
++ Add timeout middleware to server (to prevent hanging connections) — [this one](https://github.com/gin-contrib/timeout)
+  seems nice
++ Improve CI/CD pipeline by adding continuous deployment to Cloud Run.
+
+## My question is not listed here, what should I do?
+
+Don't hesitate to ask me directly, I will be happy to answer any questions you might have.
+The best way to reach me is via telegram: @danielorlov.