Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/professional' into professional
Browse files Browse the repository at this point in the history
  • Loading branch information
nbgitpuller committed Jun 10, 2020
2 parents a2a6a6f + ad5a5e8 commit 848d733
Showing 1 changed file with 84 additions and 58 deletions.
142 changes: 84 additions & 58 deletions 006-latency/Latency.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
}
},
"source": [
"Jupyter is all well and good, but sometimes what we want are simple Python scripts and a traditional scientific interface. This where Spyder comes in..."
"Jupyter is all well and good, but sometimes what we want are simple Python scripts and a traditional project layout. This where VSCode comes in..."
]
},
{
Expand All @@ -37,9 +37,23 @@
}
},
"source": [
"## Starting Spyder\n",
"## Starting VSCode\n",
"\n",
"Hit the \"Activites\" icon in the top-right and type \"spyder\"<Return>"
"At a command line, create a new folder somewhere under your home directory, and clone the Python course into it:\n",
"\n",
" git clone https://github.com/flaxandteal/python-course\n",
" cd python-course\n",
" git checkout professional\n",
" \n",
"Head to the 006-latency folder:\n",
"\n",
" cd 006-latency\n",
"\n",
"Run VSCode:\n",
"\n",
" code .\n",
" \n",
"Open the `006-latency` directory from the file menu."
]
},
{
Expand All @@ -50,9 +64,9 @@
}
},
"source": [
"[Spyder](https://www.spyder-ide.org/) has been growing over the last couple years and is quite well set up for scientific Python...\n",
"Open `network_test_client.py` from the panel on the left. If this is your first time opening a Python file in VSCode, it may ask (bottom right) if you want to install the Python plugin. Go ahead and do so.\n",
"\n",
"You will see on the left, a large editor window like any other IDE. On the bottom right, is an IPython notebook. This is really just the text version (and origin of) Jupyter. You can use it that same way, modulo interactive graphs and plotting (which you can instead write to a file, say). The top right window provides a handy way of accessing object documentation."
"If you do not already have a terminal at the bottom of the screen, go ahead and open one now (Ctrl+Shift+# or via Terminal menu). This will put you in the same directory as the one you opened."
]
},
{
Expand All @@ -78,17 +92,6 @@
"Virtualenv provides a means of sandboxing a Python installation, that is, installing a particular version of Python and all the modules you want in a directory, and using everything from there."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"source": [
"* Press the Windows key and start typing \"Terminal\" - when you see a black computer screen icon, click it."
]
},
{
"cell_type": "markdown",
"metadata": {
Expand All @@ -97,32 +100,36 @@
}
},
"source": [
"Put up your arrows when done!"
"`venv` is a Python for setting up virtual environments - where you can create an \"environment\" and install completely separate packages from the system or any other environment. Basically, it's a directory with a load of Python modules"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
"slide_type": "slide"
}
},
"source": [
"`venv` is a Python for setting up virtual environments - where you can create an \"environment\" and install completely separate packages from the system or any other environment. Basically, it's a directory with a load of Python modules"
"* In the `006-latency` directory execute **`python3 -m venv .venv`**\n",
" * This creates a new subdirectory called \".venv\" - note that it will be hidden by default when exploring your directory, as it starts with a dot\n",
" \n",
"VSCode may prompt you, at this point, to use the new environment for your Workspace. If so, say Yes.\n",
"\n",
"If not, press `Ctrl+Shift+P` and you will see a command input textbox. Start typing `Python: Select Interpreter` and select it when it appears.\n",
"\n",
"You should see a line with `{'.venv': venv}` - pick it."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"metadata": {},
"source": [
"* In the `python-course` directory execute **`python3 -m venv env`**\n",
" * This creates a new subdirectory called \"env\"\n",
"* Execute **`source env/bin/activate`**\n",
" * This loads the new environment"
"If you do change environments, you will still have to close the Terminal and open a new one, before running Python, pylint, pytest, etc. in there reaches the same versions as VSCode is using. Alternatively (and if not using VSCode), you can run the command:\n",
"\n",
" . .venv/bin/activate\n",
" \n",
"and you Terminal will now use project-specific versions of Python and its libraries."
]
},
{
Expand All @@ -133,9 +140,7 @@
}
},
"source": [
"On Windows, this is basically the same but without `source` and swapping slash direction. Mac is the same.\n",
"\n",
"However, I will recommend a tool called Anaconda, which can vastly simplify Python configuration on Windows and Mac - as I only work with Linux at present, I haven't needed it, but it is highly recommended for those systems. Anaconda is a \"distribution\" of Python, which bundles a version of the Python interpreter, a whole load of modules and a similar, but slightly different, way of working with environments. Instead of `pip`, it uses a package manager called `conda`."
"Anaconda is a popular distribution, especially in data science - for the moment, we'll stick to the generic tools (pip, etc.), but if you are having trouble with those, it can vastly simplify Python configuration on Windows and Mac. Anaconda is a \"distribution\" of Python, which bundles a version of the Python interpreter, a whole load of modules and a similar, but slightly different, way of working with environments. Instead of `pip`, it uses a package manager called `conda`."
]
},
{
Expand All @@ -146,8 +151,13 @@
}
},
"source": [
"* Run **`pip3 install sympy spyder`**\n",
" * Notice on the third-last line, it is working inside `env`"
"* Run **`pip3 install sympy pylint`**\n",
"* Run **`which python3`**\n",
"* Notice that the version of Python that runs by default is within the `.venv` subdirectory\n",
"\n",
"We have also put those into the `requirements.txt` file, so you could also run\n",
"\n",
"* **`pip3 install -r requirements.txt`**"
]
},
{
Expand All @@ -158,7 +168,7 @@
}
},
"source": [
"`pip` is Python's home-grown package manager. `pip3` is the Python3 version, which is the one we want in this course. It installs modules from [PyPI](https://pypi.python.org/pypi), the official online repository for modules. It is similar to, say, *packagist* for PHP, *rubygems* for Ruby or *Hackage* for Haskell. "
"`pip` is Python's home-grown package manager. It installs modules from [PyPI](https://pypi.python.org/pypi), the official online repository for modules. It is similar to, say, *packagist* for PHP, *rubygems* for Ruby or *Hackage* for Haskell. "
]
},
{
Expand All @@ -181,18 +191,7 @@
}
},
"source": [
"We want to use Spyder within the virtual environment. This means we have to start it from the terminal. To do so, type ``spyder3 & <Return>``. The ampersand (&) tells Linux that we want to continue entering commands while Spyder is running. Go back to the terminal and hit ``<Return>`` once or twice and you should see the normal prompt waiting for more input."
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "notes"
}
},
"source": [
"Since many of you are computer scientists, and I'm not, I am going to to take a risk and theme this session on measuring latency between a server and client. Please ignore the simplistic methodology - if you find a few minutes spare, feel free to improve the approach! However, the aim here is to show you some tools, so don't worry too much about the numbers.\n",
"Since some of you may have a CS background and I do not, I am going to to take a risk and theme this session on measuring latency between a server and client. Please ignore the simplistic methodology - if you find a few minutes spare, feel free to improve the approach! However, the aim here is to look at the code and algorithms.\n",
"\n",
"More broadly, for those who are coming from an engineering end, I can say from experience that if you start to scale up your number of simulations, you will be looking for bottlenecks and profiling. There are better, tailored utilities for doing so, but this should give you an idea about how to think about the problem."
]
Expand All @@ -211,13 +210,16 @@
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "fragment"
}
},
"metadata": {},
"source": [
"Return to Spyder and open up **`006-latency/network_test_client.py`**"
"On the right hand side, you will see an icon of a ladybird-like bug. Click it, and notice that the left-hand side changes to a set of panels:\n",
"\n",
"* Variables\n",
"* Watch\n",
"* Call Stack\n",
"* Breakpoints\n",
"\n",
"At the top of them, slightly to the left, is a green play arrow - press it. You will be asked how you wish to debug the current file. Select a `Python File` debug configuration."
]
},
{
Expand Down Expand Up @@ -357,7 +359,7 @@
}
},
"source": [
"I mentioned earlier that `try-except` is your friend. Bear in mind, if you highlight several lines in Spyder, Tab will indent them - Shift+Tab deindents.\n",
"I mentioned earlier that `try-except` is your friend. Bear in mind, if you highlight several lines in VSCode, Tab will indent them - Shift+Tab deindents.\n",
"\n",
"If we get a socket error from anywhere inside the `with` (including our test routine from earlier), it will get caught here. In practice this is only being used to inject an extra logging line, but it illustrates the point. Note that we aren't catching the `IOError` I mentioned earlier. To do so, you can add an extra except clause, or turn \"`OSError as e`\" into \"`(IOError, OSError) as e`\", if we are happy to use the same one. Note that you do need the tuple parens (parens == parentheses)."
]
Expand Down Expand Up @@ -728,7 +730,6 @@
},
"source": [
"# Challenge\n",
"## Stars up!\n",
"\n",
"Combine our first code into the `run` function to produce a script that tests 10 times (timeit arg) from each of 10 threads.\n",
"\n",
Expand Down Expand Up @@ -988,10 +989,12 @@
},
{
"cell_type": "markdown",
"metadata": {},
"metadata": {
"hideOutput": false
},
"source": [
"# Challenge - Ease of extension\n",
"## It is easier to extend good code than PhD funding"
"## It is easier to extend good code than R&D funding"
]
},
{
Expand Down Expand Up @@ -1031,9 +1034,32 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Final Challenge - Combination\n",
"# Challenge - Testing"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Install pytest using `pip`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Switch `test_runner_can_generate_callers` to test with real threads, but faked (\"monkey-patched\") `run_caller` methods."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In pairs, have a look at creating tests for:\n",
"\n",
"Create a system for visualizing the data from the network client. Try to make it flexible, so you can increase or decrease the number of latency tests. Experiment with a few different ways of visualizing the data - explore the matplotlib and bokeh docs. Post anything interesting you find on Etherpad!"
"* test_config : `get_config`, `ConfigManager.get` and `ConfigManager.load`\n",
"* test_client : `HttpEchoClient.create_connection`\n",
"* test_runner : `ParallelRunner.run_caller`"
]
}
],
Expand Down

0 comments on commit 848d733

Please sign in to comment.