(ARCHIVED) Week 5 Notes

#Working on the Server Side In week one, we built static file servers using Servi.js and Node.js. That is the minimum we need in order to get our HTML, CSS, and Javascript to a web browser. So far in this course, we've been mostly concerned with what happens once things are loaded in the browser. This week, we're going to focus on what we can do before our pages even get to the browser. The server will read the browser's request and decide what to send back based on the info it gets.

For instance, we might decide the path to a user's profile in our application is mysite.com/profile/USERNAME, so that my profile URL might be mysite.com/profile/robyn, and Shawn's URL could be mysite.com/profile/shawn. Without server side programming, we'd have to build a separate HTML page for each user. In the way we've been doing things until now, if we wanted 2 pages, we'd have to create them each by hand and upload them: robyn.html and shawn.html, for example. The two pages would likely look exactly the same, except that one would have Shawn's information and one would have mine. With server-side programming, we can make just one HTML page and then fill in the content based on which profile the user requests.

Working on the server side, we're going to intercept the request, decide what to send to the user, and then send it out. Once it gets to the browser, it will look like any other HTML/CSS/Javascript page, just as if we'd created robyn.html and uploaded it to the server. The difference is, the page is created programmatically, and we don't have to laboriously create each page by hand.

Tools and Workflow

Servi without the IDE. Previously, we used servi.js with the Servi IDE. In the exercises this week, we're still using servi.js, but we're going to edit javascript in Sublime (or your text editor of choice) and skip the IDE. The toolset as we're working will be:

Locally, on your computer

Text editor (Sublime or other), where you write the Javascript. For example, you might create a file called server.js.
Terminal, where you run the javascript using node. For example: node server.js
A browser, where you test the server. For example, if your server.js script runs a server on port 3000, you'd check the URL http://localhost:3000 in your browser.

Once you get things running on your computer, you'll then upload your javascript to Digital Ocean and run it there. You'll also use the terminal to run the script there, but you'll need to log in to your Digital Ocean server with ssh first. The workflow looks like this:

Remotely, on your Digital Ocean server

Using Cyberduck, upload your Javascript file (server.js or whatever you named it) to your Digital Ocean droplet. You can put it anywhere you like, but for organization's sake, you may want to create separate folders for separate projects, for example, mySocialNetworkServer/server.js or testServerNov15/server.js
In Terminal, ssh to your droplet (as described here ) and navigate to the directory that contains your Javascript file -- server.js or whatever you named it.

Run node NAME_OF_YOUR_JAVASCRIPT_FILE.js. Servi will tell you which port it is running on. If you did not set a port explicitly, it will usually be port 3000.
Test in the browser, just as you did locally. This time, instead of localhost, you'll go to your IP address. For example, http://123.456.78:3000

Installing Node and installing Servi

Because you are working on two computers -- your physical Mac laptop (or whatever computer you may have) and the Digital Ocean server in the cloud -- you need to make sure both Node and Servi are installed in both places. So, we've got 2 computers and 2 pieces of software:

Things to install:

Node
Servi

Places to install them:

Laptop
Digital Ocean cloud server

Node. The Digital Ocean droplets we create have node installed already, so we don't have to do anything there. To check if Node is installed on your Mac, open Terminal and type which node. If you get a path back, something like, /usr/local/bin/node, then it's installed. If you get nothing, that means you'll need to install it. Download and install it.

Servi. Servi is a module for Node. Every Node project uses any number of modules in order to run. In general, much of the module management in Node happens behind the scenes, but there is some initial set up to start using a new module.

Package Manager. Node has a built-in tool to download and install modules, called Node Package Manager, or npm. We'll use npm to install the Servi module. NPM lets you decide whether you want to install a module globally or on a per-project basis (see the note below for more on that). Since we'll be using servi a lot, we'll install it globally. In Terminal, to install Servi globally, type:

npm install servi -g

Because you'll be working in two different systems, you'll need to do this in both places -- your computer and your Digital Ocean Server. Both happen in Terminal. For Digital Ocean, you'll need to ssh into your account first.

Note on Global vs. Per-project. In Node, there are two ways to use modules -- on a per-project basis, or globally. If you install module files on a per-project basis (aka, locally), all the modules you need will be stored in a directory called node_modules inside the same directory as your node javascript file. It's much easier, at least for the work we're doing, to install the modules globally, meaning that instead of creating a new folder full of modules every time, the module files will be stored in a central location on your system, and out of your way. With npm, using the -g flag next to install command will install a module globally.

Using Servi.js

Routing

In servi, we can set up the URLs our application will use. This is called routing. In servi, you set the name of the URL, or route, and then the name of a function to run when that URL is visited.

route('/one',handleOne);
route('/two',handleTwo);
route('/buckle/my/shoe',buckleMyShoe);
		
function handleOne(request) {
    console.log("one");
    request.respond("thanks one");
}			

function handleTwo(request) {
    console.log("two");
    request.respond("thanks two");
}

function buckleMyShoe(request) {
    console.log("buckle my shoe");
    request.respond("buckle my shoe");
}

Each function called in route takes a request parameter, which contains information about the user's request and lets you respond to the request, using the .respond() method.

The exciting part comes when the route handler can take in additional information and give a customized response based on that information.

route('/profile/:username',showProfile);

function showProfile(request){
    request.respond("All about " + request.params.username);
}

Define a variable in a route by using a colon (:) in front of the variable name. Then retrieve the value of the variable from the request object using request.params.VARIABLE_NAME.

See the following scripts in the repo for examples using parameter variables:

Forms

#####The query string In addition to accessing variables you set up in route(), the request.params object will also contain any data that comes in through the query string in the URL. You've seen examples of the query string in some of the external APIs we used. For example, the Instagram API uses the query string in its URL to set the client id and the latitude/longitude in a location search: https://api.instagram.com/v1/media/search?lat=48.858844&lng=2.294351&client_id=abc123abc12345.

In servi, the URL mysite.com/info/person?city=Brooklyn&name=Jim&school=NYU would provide the variables city,name, and school as request.params.city, request.params.name, etc.

#####GET Forms can be used to create query strings. When a form uses the method is GET, the content will show up in the URL submitted to the server.

<form method="GET" action="/somescript">
    <input type="text" name="name" value="Your Name">
    <input type="text" name="city" value="Your City">
    <input type="submit" name="submit" value="Submit">
</form>

When a form is submitted using the GET method, the browser goes to the URL set in the "action" attribute and tacks on the form values in the query string. The form above, when submitted with the values "Jane" and "Brooklyn", would go to the url whateversite.com/somescript?name=Jane&city=Brooklyn&submit=Submit.

In Servi, we can access the data with:

route('/somescript',handleQuery);

function handleQuery(request){
    var response = "The name submitted was: " + request.params.name + "\n";
    response += "The city submitted was: " + request.params.city;
    request.respond(response);
}

See the following folder in the repo for a form example using GET: https://github.com/robynitp/networkedmedia/tree/master/10-server-side/03-servi-form-get

POST

If we change the form "method" to "POST" the content is sent in a manner that we can't see in the URL. You would use this method for information that shouldn't be submitted more than once (such as entering a comment) or that shouldn't be visible in the URL such as username/password:

<form method="POST" action="/somescript">
	<input type="text" name="name" value="Your Name">
	<input type="text" name="city" value="Your City">
	<input type="submit" name="submit" value="Submit">
</form>

In Servi, the data in a POST query comes in via the request.fields object:

route('/somescript',handleQuery);

function handleQuery(request){
    var response = "The name submitted was: " + request.fields.name + "\n";
    response += "The city submitted was: " + request.fields.city;
    request.respond(response);
}

See the following folder in the repo for a form example using POST: https://github.com/robynitp/networkedmedia/tree/master/10-server-side/04-servi-form-post

The Servi database

####Saving Data

Servi has a built in database which uses JSON. (For the record, it uses NeDB, a node database library).

// Create or use the existing database
var infoDB = useDatabase("namescities");

route('/somescript',handleQuery);

function handleQuery(request)
{
	// request.params.name
	// request.params.city
	var response = "The name submitted was: " + request.fields.name + "\n";
	response += "The city submitted was: " + request.fields.city;
	
	// Create a JSON object to hold the data
	var dataToSave = {name: request.fields.name, city: request.fields.city};
	
	// Save the data to the database
	infoDB.add(dataToSave);
	
	request.respond(response);
}

####Retrieving Data

To pull data out of the database:

// Create or use the existing database
var infoDB = useDatabase("namescities");

route('/viewdata',viewData);

function viewData(request) {
	infoDB.getAll(gotData);    

	function gotData(theData) {
		var response = "";
		for (i = 0; i < theData.length; i++) {
			response += theData[i].name + " lives in " + theData.city + "\n";
		}
		request.respond(response);
	}
}

See this Servi database example in the repo: https://github.com/robynitp/networkedmedia/blob/master/10-server-side/06-servi-db/server.js

For more info on using the Servi database, see the Servi documentation.

Templating

When you want to create an HTML page once and use it as template for different pieces of data, as in the user profile example above, Servi provides the render() function. By default, Servi uses the EJS template rendering system. Check out the EJS documentation for more complete details.

The render() function takes two parameters: the HTML template, and an object containing the data that will go into the template.

var mydata = {
    title: "Things You Never Knew",
    date: "November 24, 2014",
    articleText: "Donec diam nibh, imperdiet ac lacus sit amet, accumsan tincidunt urna."
}
request.render("mytemplate.html",mydata);

In the HTML template, create placeholders for the data, using the same names as the properties of the object -- in this example, we have title, date, and articleText. So the HTML template might look like:

<html>
 <head>
    <title><%= title %></title>
 </head>
 <body>
    <h1>New York Daily News</h1>

    <h2><%= title %></h2>
    <h3><%= date %></h3>
    <p><%= articleText %></p>  
 </body>
</html>

If a template variable contains an array or object, you can use a loop to iterate over elements. For example, the object below contains an array called people:

var data = {
	people: [
	    {
	        "name": "Brad",
	        "city": "London"
	    },
	    {
	        "name": "Jill",	
	        "city": "Tokyo"
	    },
	    {
	        "name": "Kate",
	        "city": "New York"
	    }
	]
};

Then, using request.render("my_template.html",data);, you could hook it up to a template that contained a loop, like this:

<h1>Where people live</h1>
<!-- Embed a loop in the template tags -->
<% for (i = 0; i < people.length; i++) { %>
        
    <li><%= people[i].name %> lives in <%= people[i].city %></li>

<% } %> <!-- end the for loop -->