-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathdeployment.html
158 lines (156 loc) · 6.73 KB
/
deployment.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
<!DOCTYPE html>
<html lang="en">
<head>
<title>COLD (Controlled Object List and Datum (Concept))</title>
<link href='https://fonts.googleapis.com/css?family=Open+Sans' rel='stylesheet' type='text/css'>
<link rel="stylesheet" href="https://caltechlibrary.github.io/css/site.css">
</head>
<body>
<header>
<a href="http://library.caltech.edu" title="link to Caltech Library Homepage"><img src="https://caltechlibrary.github.io/assets/liblogo.gif" alt="Caltech Library logo"></a>
</header>
<nav>
<ul>
<li><a href="/">Home</a></li>
<li><a href="index.html">README</a></li>
<li><a href="LICENSE">LICENSE</a></li>
<li><a href="INSTALL.html">INSTALL</a></li>
<li><a href="user_manual.html">User Manual</a></li>
<li><a href="about.html">About</a></li>
<li><a href="https://github.com/caltechlibrary/cold">GitHub</a></li>
</ul>
</nav>
<section>
<h1 id="deploying-cold">Deploying <strong>cold</strong></h1>
<p>Deploying cold on a remote system requires manual setup. You will
need the following software to successfully build and deploy.</p>
<ul>
<li>Deno >= 2.0.6 (for compiling COLD)</li>
<li>Dataset >= 2.1.23</li>
<li>Pandoc >= 3.1</li>
<li>GNU Make</li>
<li>Git</li>
<li>Tmux highly recommended</li>
<li>eLinks or Lynx recommended (to test form the console)</li>
</ul>
<h2 id="deployment-steps">Deployment steps</h2>
<p>My current recommendation is the following.</p>
<ol type="1">
<li>Setup the directory to hold he web application if it doesn’t
exist.</li>
<li>Clone the repository, e.g. clone to <code>/Sites/cold</code> and
change into the repository directory</li>
<li>Run Deno tasks <code>setup</code> (if the collections don’t exist)
and <code>build</code> to build the binary for the cold service</li>
<li>Copy <code>cold.service-example</code> to <code>cold.service</code>,
edit it and move to <code>/etc/systemd/system/</code></li>
<li>Copy <code>cold_api.service-example</code> to
<code>cold_api.service</code>, edit it and and move to
<code>/etc/systemd/system/</code></li>
<li>Reload the systemd daemon,
<code>sudo systemctl daemon-reload</code></li>
<li>Enable the services (only needed the first time, may return a
warning about symbolic link)
<ol type="a">
<li><code>sudo systemctl enable cold.service</code></li>
<li><code>sudo systemctl enable cold_api.service</code></li>
</ol></li>
<li>Start the services using <code>systemctl</code> in the usual way
<ol type="a">
<li><code>sudo systemctl start cold.service</code></li>
<li><code>sudo systemctl start cold_api.service</code></li>
</ol></li>
<li>Test web services using eLinks. If you get a gateway error it means
datasetd isn’t running correctly in port 8111. Debug with curl,
systemctl status, journalctl.</li>
<li>Make sure the <code>htdocs/rpt</code> directory has the correct
ownership and sticky bit set so that reports can be run from a cronjob
and from the reports UI and not get “stuck”.</li>
</ol>
<p>You can configure Apache to reverse proxy to the cold service running
on port 8111 where it should enforce access control.</p>
<p>For a good description of how to setup new systemd services the
Debian (works with Ubuntu too) way see <a
href="https://wiki.debian.org/systemd/Services"
class="uri">https://wiki.debian.org/systemd/Services</a>.</p>
<p>Here’s an example of the shell session based on the above list. I’m
assuming the user/group you’re running things under is “www-data”.</p>
<pre class="shell"><code>ssh apps.example.edu
sudo mkdir -p /Sites/
sudo chgrp www-data /Sites/
sudo chmod 775 /Sites/
cd /Sites
git clone [email protected]:caltechlibrary/cold
cd cold
deno task setup
deno task build
cp cold.service-example cold.service
nano cold.service
sudo mv cold.service /etc/systemd/system/
cp cold_api.service-example cold_api.service
nano cold_api.service
sudo mv cold_api.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable cold.service
sudo systemctl enable cold_api.service
sudo systemctl start cold.service
sudo systemctl start cold_api.service
elinks http://localhost:8111
sudo chown -R $USER:$USER htdocs/rpt
chmod +t htdocs/rpt</code></pre>
<p>NOTE: The TypeScript services need to be compile before running them
with Systemd.</p>
<h2 id="apache-2-and-shibboleth">Apache 2 and Shibboleth</h2>
<p><strong>COLD</strong> is designed to be a reverse proxy target. Using
Apache you need to to include the following code in the main host
definition.</p>
<pre><code>#<!-- cold -->
ProxyPreserveHost On
Redirect "/cold" "/cold/"
ProxyPassMatch "^/cold/(.*)" "http://localhost:8111/$1"
ProxyPassReverse "/cold/" "http://localhost:8111/"
#<!-- cold admin -->
<Location /cold/>
AuthType shibboleth
ShibRequestSetting requireSession 1
#require valid-user
</Location>
#<!-- end cold --></code></pre>
<h2
id="example-data-migration-from-spreadsheets-for-caltechpeople">Example
data migration from spreadsheets for CaltechPEOPLE</h2>
<p>This is a note for migrating data from our historic spreadsheet for
CaltechPEOPLE. Once cold is installed and you’ve created the empty
dataset collections (e.g. people.ds) you can use two tools to populate
the collection and set the <code>include_in_feeds</code> property.</p>
<p>Steps:</p>
<ol type="1">
<li>Set the environment variable FEEDS_BASE_DIR to point to where feeds
staging is deployed.</li>
<li>Copy people.csv and groups.csv from feeds.library.caltech.edu and
save then as <code>people_final.csv</code> and
<code>groups_final.csv</code>. Copy edit the files if necessary
(e.g. remove duplicate rows)</li>
<li>Copy the directory names for people on feeds.library.caltech.edu and
render this as a <a href="in_feeds.csv-example">single CSV column</a>
with the heading <code>clpid</code>.</li>
<li>Clear test data from <code>people.ds</code> with SQLite3 cli.</li>
<li>Run <code>ds_importer</code></li>
<li>Run <code>set_include_in_feeds</code></li>
</ol>
<pre class="shell"><code>FEEDS_BASE_DIR="/Sites/feeds"
scp "library.example.edu:$FEEDS_BASE_DIR/people.csv" people_final.csv
echo "clpid" >in_feeds.csv
ssh library.example.edu "ls -1 -d $FEEDS_BASE_DIR/htdocs/people/* | cut -d / -f 6" >>in_feeds.csv
./bin/ds_importer people.ds people_final.csv
./bin/set_include_in_feeds people.ds in_feeds.csv</code></pre>
</section>
<footer>
<span>© 2022 <a href="https://www.library.caltech.edu/copyright">Caltech Library</a></span>
<address>1200 E California Blvd, Mail Code 1-32, Pasadena, CA 91125-3200</address>
<span><a href="mailto:[email protected]">Email Us</a></span>
<span>Phone: <a href="tel:+1-626-395-3405">(626)395-3405</a></span>
</footer>
</body>
</html>