-
Notifications
You must be signed in to change notification settings - Fork 4
/
Copy path00_UnixNotes.txt
178 lines (100 loc) · 5.33 KB
/
00_UnixNotes.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
---------------------------------------------------------------------
Astronomy 300 - Week 0 Notes - Command Line Data Exploration
---------------------------------------------------------------------
Open up a terminal on one the astrolab machines (right click on the
background and select Open Terminal). First thing to do is to run the
setup file. The "%" is the Unix prompt, you do not type this character.
Get the setup file:
% cp ~pikaia/setup . make sure to type the "."
% tcsh setup
Lets get some files to work with:
% git clone https://github.com/UWashington-Astro300/Astro300-A17.git
% cd Astro300-A17 change directory
% pwd Show the current directory path.
% ls list the files in a directory
---------------------------------------------------------------------
% cd MyData change to the Data directory
% cat BrightStars.csv
% more BrightStars.csv "q" to quit
% less BrightStars.csv "q" to quit
% cp BrightStars.csv junk.dat Copy files
% mv junk.dat junque.dat Rename files
% rm junque.dat Delete file (NO RECOVERY!)
---------------------------------------------------------------------
wildcards (*) and (?)
* - Matches ANY (0 or more) character or characters in a filename.
? - Matches ANY ONE character in a filename.
% ls B*.csv will find BrightStars.csv
% ls B?.csv will NOT find BrightStars.csv
Be very careful with "*". It can ruin your life! For example "rm *"
will wipe out all of your files in the current directory, no questions
asked, no chance of recovery.
---------------------------------------------------------------------
Ctrl+a jumps to the beginning of command-line
Ctrl+e jumps to the end of command-line
[TAB] Completion
% less R[TAB] Hit [TAB] again to show all options
[UP] [DOWN] arrows - command history
% history Shows history of commands
---------------------------------------------------------------------
Line Numbers
% nl Jabber.txt Shows line number
% nl -ba Jabber.txt including blank lines (more useful)
---------------------------------------------------------------------
Chopping Streams (head and tail)
% head BrightStars.csv first 10 lines
% tail BrightStars.csv last 10 lines
% head -20 BrightStars.csv first 20 lines
% tail -20 BrightStars.csv last 20 lines
---------------------------------------------------------------------
Diverting Streams (> and >>)
% head -13 10Bright.csv > header.txt (>) crates file, overwriting old
% ls
% wc 10Bright.csv number of lines, words, and bytes
% wc header.txt
% tail -11 BrightStars.csv >> 10Bright.csv (>>) appends data to file
% wc 10Bright.csv
---------------------------------------------------------------------
Connecting Streams - pipes (|)
% cat 10Bright.csv | grep 'e' <- grep matches thing in ' '
% cat 10Bright.csv | grep 'e' | wc
% cat 10Bright.csv | grep -v '^\#' | grep 'e'
ignore lines beginning with # - This is VERY useful
---------------------------------------------------------------------
Sorting Streams - sort
% cat 10Bright.csv | grep -v '^\#' | sort -t ',' -k 2 Alphabetical sort (default)
% cat 10Bright.csv | grep -v '^\#' | sort -t ',' -k 1 Not good for numbers!
% cat 10Bright.csv | grep -v '^\#' | sort -t ',' -g -k 1 Numerical sort (very different)
% cat 10Bright.csv | grep -v '^\#' | sort -t ',' -g -r -k 5 Reverse sort
-------------------------------------------------------------
AWK - Aho, Weinberger, & Kernighan
% cat 10Bright.csv | grep -v '^\#' | awk -F, '{print $2}' print 2nd column
% cat 10Bright.csv | grep -v '^\#' | awk -F, '{print $0}' Same as % cat 10Bright.csv
% cat 10Bright.csv | grep -v '^\#' | awk -F, '{print $5,$1}'
% cat 10Bright.csv | grep -v '^\#' | awk -F, '{OFS=","}{print $5,$1}'
-------------------------------------------------------------
1E3 = 1000 2^3 = 2*2*2 = 8 380%360 = 20
== equal to && and
!= not equal to || or
> greater than % Mod
< less than ++ Increment
>= greater than or equal to -- Decrement
<= less than or equal to += i += 6 (i = i + 6)
sqrt() Square root int() Integer part of argument
log() ln exp() e^x
-------------------------------------------------------------
Print all lines where the value of column 4 > 0.0:
% cat 10Bright.csv | grep -v '^\#' | awk -F, '{if ($4 > 0.0) print $0}'
Print all lines where the value of column 5 is between 0.0 and 1.0:
% cat 10Bright.csv | grep -v '^\#' | awk -F, '{if ($5>0.0 && $5<1.0) print $0}'
Print all lines where the value of column 2 contains "A" (anywhere in the column).
% cat 10Bright.csv | grep -v '^\#' | awk -F, '{if ($2~/A/) print $0}'
You can append '| wc' to the end of each of the above to get a count
-------------------------------------------------------------
A simple text editor: The astrolab machines have a variety of text
editors available. If you are already use to one, fell free to it. If
you have never used a text editor on these machine, the good one to
start with is gedit.
% gedit &
Make sure to type the "&" on the line above. This places the editor in
the background so your terminal is free to do other tasks.