+csvcleaner normalizes a CSV file based on the options selected. It helps
+to address issues like variable number of columns, leading/trailing
+spaces in columns, and non-UTF-8 encoding issues.
+
+
+By default input is expected from standard in and output is sent to
+standard out (errors to standard error). These can be modified by
+appropriate options. The csv file is processed as a stream of rows so
+minimal memory is used to operate on the file.
+
+
+OPTIONS
+
+
+
+-help
+
+
+display help
+
+
+-license
+
+
+display license
+
+
+-version
+
+
+display version
+
+
+-verbose
+
+
+write verbose output to standard error
+
+
+-comma
+
+
+if set use this character in place of a comma for delimiting cells
+
+
+-comment-char
+
+
+if set, rows starting with this character will be ignored as comments
+
+
+-fields-per-row
+
+
+set the number of columns to output right padding empty cells as needed
+
+
+-i, -input
+
+
+input filename
+
+
+-left-trim
+
+
+left trim spaces on CSV out
+
+
+-o, -output
+
+
+output filename
+
+
+-output-comma
+
+
+if set use this character in place of a comma for delimiting output
+cells
+
+
+-quiet
+
+
+suppress error messages
+
+
+-reuse
+
+
+if false then a new array is allocated for each row processed, if true
+the array gets reused
+
+
+-right-trim
+
+
+right trim spaces on CSV out
+
+
+-stop-on-error
+
+
+exit on error, useful if you’re trying to debug a problematic CSV file
+
+
+-trim, -trim-spaces
+
+
+trim spaces on CSV out
+
+
+-trim-leading-space
+
+
+trim leading space from field(s) for CSV input
+
+
+-use-crlf
+
+
+if set use a charage return and line feed in output
+
+
+-use-lazy-quotes
+
+
+use lazy quotes for CSV input
+
+
+
+EXAMPLES
+
+
+Normalizing a spread sheet’s column count to 5 padding columns as needed
+per row.
+
+csvfind processes a CSV file as input returning rows that contain the
+column with matched text. Columns are counted from one instead of zero.
+Supports exact match as well as some Levenshtein matching.
+
+
+OPTIONS
+
+
+
+-help
+
+
+display help
+
+
+-license
+
+
+display license
+
+
+-version
+
+
+display version
+
+
+-allow-duplicates
+
+
+allow duplicates when searching for matches
+
+
+-append-edit-distance
+
+
+append column with edit distance found (useful for tuning levenshtein)
+
+
+-case-sensitive
+
+
+perform a case sensitive match (default is false)
+
+
+-col, -cols
+
+
+column to search for match in the CSV file
+
+
+-contains
+
+
+use contains phrase for matching
+
+
+-d, -delimiter
+
+
+set delimiter character
+
+
+-delete-cost
+
+
+set the delete cost to use for levenshtein matching
+
+
+-i, -input
+
+
+input filename
+
+
+-insert-cost
+
+
+set the insert cost to use for levenshtein matching
+
+
+-levenshtein
+
+
+use levenshtein matching
+
+
+-max-edit-distance
+
+
+set the edit distance thresh hold for match, default 0
+
+
+-nl, -newline
+
+
+include trailing newline from output
+
+
+-o, -output
+
+
+output filename
+
+
+-quiet
+
+
+suppress error messages
+
+
+-skip-header-row
+
+
+skip the header row
+
+
+-stop-words
+
+
+use the colon delimited list of stop words
+
+
+-substitute-cost
+
+
+set the substitution cost to use for levenshtein matching
+
+
+-trim-leading-space
+
+
+trim leadings space in field(s) for CSV input
+
+
+-trimspace, -trimspaces
+
+
+trim spaces around cell values before comparing
+
+
+-use-lazy-quotes
+
+
+use lazy quotes on CSV input
+
+
+
+EXAMPLES
+
+
+Find the rows where the third column matches “The Red Book of Westmarch”
+exactly
+
+
csvfind -i books.csv -col=2 "The Red Book of Westmarch"
+
+Find the rows where the third column (colums numbered 1,2,3) matches
+approximately “The Red Book of Westmarch”
+
+
csvfind -i books.csv -col=2 -levenshtein \
+ -insert-cost=1 -delete-cost=1 -substitute-cost=3 \
+ -max-edit-distance=50 -append-edit-distance \
+ "The Red Book of Westmarch"
+
+In this example we’ve appended the edit distance to see how close the
+matches are.
+
+csvjoin outputs CSV content based on two CSV files with matching column
+values. Each CSV input file has a designated column to match on. The
+values are compared as strings. Columns are counted from one rather than
+zero.
+
+
+OPTIONS
+
+
+
+-help
+
+
+display help
+
+
+-license
+
+
+display license
+
+
+-version
+
+
+display version
+
+
+-allow-duplicates
+
+
+allow duplicates when searching for matches
+
+
+-case-sensitive
+
+
+make a case sensitive match (default is case insensitive)
+
+
+-col1
+
+
+column to on join on in first CSV file
+
+
+-col2
+
+
+column to on join on in second CSV file
+
+
+-contains
+
+
+match columns based on csv1/col1 contained in csv2/col2
+
+
+-csv1
+
+
+first CSV filename
+
+
+-csv2
+
+
+second CSV filename
+
+
+-d, -delimiter
+
+
+set delimiter character
+
+
+-delete-cost
+
+
+deletion cost to use when calculating Levenshtein edit distance
+
+
+-in-memory
+
+
+if true read both CSV files
+
+
+-insert-cost
+
+
+insertion cost to use when calculating Levenshtein edit distance
+
+
+-levenshtein
+
+
+match columns using Levensthein edit distance
+
+
+-max-edit-distance
+
+
+maximum edit distance for match using Levenshtein distance
+
+
+-o, -output
+
+
+output filename
+
+
+-quiet
+
+
+supress error messages
+
+
+-stop-words
+
+
+a column delimited list of stop words to ingnore when matching
+
+
+-substitute-cost
+
+
+substitution cost to use when calculating Levenshtein edit distance
+
+
+-trim-leading-space
+
+
+trim leading space in field(s) for CSV input
+
+
+-trimspaces
+
+
+trim spaces around cell values before comparing
+
+
+-use-lazy-quotes
+
+
+use lazy quotes for CSV input
+
+
+-verbose
+
+
+output processing count to stderr
+
+
+
+EXAMPLES
+
+
+Simple usage of building a merged CSV file from data1.csv and data2.csv
+where column 1 in data1.csv matches the value in column 3 of data2.csv
+with the results being written to merged-data.csv..
+
+csvrows converts a set of command line args into rows of CSV formated
+output. It can also be used to filter or list specific rows of CSV input
+The first row is 1 not 0. Often row 1 is the header row and csvrows
+makes it easy to output only the data rows.
+
+
+OPTIONS
+
+
+
+-help
+
+
+display help
+
+
+-license
+
+
+display license
+
+
+-version
+
+
+display version
+
+
+-d, -delimiter
+
+
+set delimiter character
+
+
+-header
+
+
+display the header row (alias for ‘-rows 1’)
+
+
+-i, -input
+
+
+input filename
+
+
+-o, -output
+
+
+output filename
+
+
+-quiet
+
+
+suppress error messages
+
+
+-random
+
+
+return N randomly selected rows
+
+
+-row, -rows
+
+
+output specified rows in order (e.g. -row 1,5,2-4))
+
+
+-skip-header-row
+
+
+skip the header row (alias for -row 2-
+
+
+-trim-leading-space
+
+
+trim leading space in field(s) for CSV input
+
+
+-use-lazy-quotes
+
+
+use lazy quotes for CSV input
+
+
+
+EXAMPLES
+
+
+Simple usage of building a CSV file one rows at a time.
+
+jsoncols provides scripting flexibility for data extraction from JSON
+data returning the results in columns. This is helpful in flattening
+content extracted from JSON blobs. The default delimiter for each value
+extracted is a comma. This can be overridden with an option.
+
+
+
+EXPRESSION can be an empty string or dot notation for an object’s path
+
+
+INPUT_FILENAME is the filename to read or a dash “-” if you want to
+explicitly read from stdin
+
+
+if not provided then jsoncols reads from stdin
+
+
+
+
+OUTPUT_FILENAME is the filename to write or a dash “-” if you want to
+explicitly write to stdout
+
+jsonmunge is a command line tool that takes a JSON document and one or
+more Go templates rendering the results. Useful for reshaping a JSON
+document, transforming into a new format, or filter for specific
+content.
+
+
+
+TEMPLATE_FILENAME is the name of a Go text tempate file used to render
+the outbound JSON document
+
+jsonrange returns returns a range of values based on the JSON structure
+being read and options applied. Without options the JSON structure is
+read from standard input and writes a list of keys to standard out. Keys
+are either attribute names or for arrays the index position (counting
+form zero). If a DOT_PATH_EXPRESSION is included on the command line
+then that is used to generate the results. Using options to can choose
+to read the JSON data structure from a file, write the output to a file
+as well as display values instead of keys. a list of “keys” of an index
+or map in JSON.
+
+
+Using options it can also return a list of values. The JSON object is
+read from standard in and the resulting list is normally written to
+standard out. There are options to read or write to files. Additional
+parameters are assumed to be a dot path notation select the parts of the
+JSON data structure you want from the range.
+
+
+DOT_PATH_EXPRESSION is a dot path stale expression indicating what you
+want range over. E.g.
+
+
+
+. would indicate the whole JSON data structure read is used to range
+over
+
+
+.name would indicate to range over the value pointed at by the “name”
+attribute
+
+
+[“name”] would indicate to range over the value pointed at by the “name”
+attribute
+
+
+[0] would indicate to range over the value held in the zero-th element
+of the array
+
+
+
+The path can be chained together
+
+
+
+.name.family would point to the value heald by the “name” attributes’
+“family” attribute.
+
+
+
+OPTIONS
+
+
+
+-help
+
+
+display help
+
+
+-license
+
+
+display license
+
+
+-version
+
+
+display version
+
+
+-d, -delimiter
+
+
+set delimiter for range output
+
+
+-i, -input
+
+
+read JSON from file
+
+
+-last
+
+
+return the index of the last element in list (e.g. length - 1)
+
-Copyright (c) 2022, Caltech All rights not granted herein are expressly
+Copyright (c) 2023, Caltech All rights not granted herein are expressly
reserved by Caltech.
+mergepath can merge the new path parts with the existing path with
+creating duplications. It can also re-order existing path elements by
+prefixing or appending to the existing path and removing the resulting
+duplicate.
+
+
+OPTIONS
+
+
+
+-help
+
+
+display help
+
+
+-license
+
+
+display license
+
+
+-version
+
+
+display version
+
+
+-a, -append
+
+
+Append the directory to the path removing any duplication
+
+
+-c, -clip
+
+
+Remove a directory from the path
+
+
+-d, -directory
+
+
+The directory you want to add to the path.
+
+
+-e, -envpath
+
+
+The path you want to merge with.
+
+
+-nl, -newline
+
+
+if true add a trailing newline
+
+
+-p, -prepend
+
+
+Prepend the directory to the path removing any duplication
+
+
+-quiet
+
+
+suppress error messages
+
+
+
+EXAMPLES
+
+
+This would put your home bin directory at the beginning of your path.
+
+range is a simple utility for shell scripts that emits a list of
+integers starting with the first command line argument and ending with
+the last integer command line argument. It is a subset of functionality
+found in the Unix seq command.
+
+
+If the first argument is greater than the last then it counts down
+otherwise it counts up.
+
+
+OPTIONS
+
+
+
+-help
+
+
+display help
+
+
+-license
+
+
+display license
+
+
+-version
+
+
+display version
+
+
+-e, -end
+
+
+The ending integer.
+
+
+-inc, -increment
+
+
+The non-zero integer increment value.
+
+
+-nl, -newline
+
+
+if true add a trailing newline
+
+
+-quiet
+
+
+suppress error messages
+
+
+-random
+
+
+Pick a range value from range
+
+
+-s, -start
+
+
+The starting integer.
+
+
+
+EXAMPLES
+
+
+Create a range of integers one through five
+
+
range 1 5
+
+Yields 1 2 3 4 5
+
+
+Create a range of integer negative two to six
+
+
range -- -2 6
+
+Yields -2 -1 0 1 2 3 4 5 6
+
+
+Create a range of even integers two to ten
+
+
range -increment=2 2 10
+
+Yields 2 4 6 8 10
+
+
+Create a descending range of integers ten down to one
+
+reldate is a small command line utility which returns the relative date
+in YYYY-MM-DD format. This is helpful when scripting various time
+relationships. The difference in time returned are determined by the
+time increments provided.
+
+
+Time increments are a positive or negative integer. Time unit can be
+either day(s), week(s), month(s), or year(s). Weekday names are case
+insentive (e.g. Monday and monday). They can be abbreviated to the first
+three letters of the name, e.g. Sunday can be Sun, Monday can be Mon,
+Tuesday can be Tue, Wednesday can be Wed, Thursday can be Thu, Friday
+can be Fri or Saturday can be Sat.
+
+
+OPTIONS
+
+
+
+-help
+
+
+display help
+
+
+-license
+
+
+display license
+
+
+-version
+
+
+display version
+
+
+-e, -end-of-month
+
+
+Display the end of month day. E.g. 2012-02-29
+
+
+-f, -from
+
+
+Date the relative time is calculated from.
+
+
+-nl, -newline
+
+
+if true add a trailing newline
+
+
+-quiet
+
+
+suppress error messages
+
+
+
+EXAMPLES
+
+
+If today was 2014-08-03 and you wanted the date three days in the past
+try–
+
+
reldate 3 days
+
+The output would be
+
+
2014-08-06
+
+TIME UNITS
+
+
+Supported time units are
+
+
+
+day(s)
+
+
+week(s)
+
+
+year(s)
+
+
+
+Specifying a date to calucate from
+
+
+reldate handles dates in the YYYY-MM-DD format (e.g. March 1, 2014 would
+be 2014-03-01). By default reldate uses today as the date to calculate
+relative time from. If you use the –from option you can it will
+calculate the relative date from that specific date.
+
+
reldate --from=2014-08-03 3 days
+
+Will yield
+
+
2014-08-06
+
+NEGATIVE INCREMENTS
+
+
+Command line arguments traditionally start with a dash which we also use
+to denote a nagative number. To tell the command line process that to
+not treat negative numbers as an “option” precede your time increment
+and time unit with a double dash.
+
+
reldate --from=2014-08-03 -- -3 days
+
+Will yield
+
+
2014-07-31
+
+RELATIVE WEEK DAYS
+
+
+You can calculate a date from a weekday name (e.g. Saturday, Monday,
+Tuesday) knowning a day (e.g. 2015-02-10 or the current date of the
+week) occurring in a week. A common case would be wanting to figure out
+the Monday date of a week containing 2015-02-10. The week is presumed to
+start on Sunday (i.e. 0) and finish with Saturday (e.g. 6).
+
+
reldate --from=2015-02-10 Monday
+
+will yield
+
+
2015-02-09
+
+As that is the Monday of the week containing 2015-02-10. Weekday names
+case insensitive and can be the first three letters of the English names
+or full English names (e.g. Monday, monday, Mon, mon).
+
+reltime provides a relative time string in the “HH:MM:SS” in 24 hour
+format.
+
+
+The notation for the relative time string is based on Go’s time duration
+string. From https://golang.google.cn/pkg/time/#ParseDuration,
+
+
+
+A duration string is a possibly signed sequence of decimal numbers, each
+with optional fraction and a unit suffix, such as “300ms”, “-1.5h” or
+“2h45m”. Valid time units are “ns”, “us” (or “µs”), “ms”, “s”, “m”, “h”.
+
+
+
+EXAMPLES
+
+
+Get the dime ninety minutes in the past.
+
+
reltime -- -90m
+
+Get the time 24 hours ago
+
+
reltime -- -24h
+
+Get the time 16 hours, 23 minutes and 4 seconds in the future.
+
+string is a command line tool for transforming strings in common ways.
+
+
+
+string length
+
+
+changing cases
+
+
+checking for prefixes, suffixes
+
+
+trimming prefixes, suffixes and cutsets (i.e. list of characters to cut)
+
+
+position, counting and replacing substrings
+
+
+splitting a string into a JSON array of strings, joining JSON a string
+arrays into a string
+
+
+
+VERB refers to the operation to performed on the supplied string(s). VER
+PARAMETERS are thsose additional terms need to complete the process
+provided by the VERB.
+
+timefmt formats the current date or INPUT_DATE based on the output
+format provided in options. The default input and output format is
+RFC3339. Formats are specified based on Golang’s time package including
+the common constants (e.g. RFC822, RFC1123).
+
+
+For details see https://golang.org/pkg/time/#Time.Format.
+
+
+One additional time layout provided by timefmt
+
+
+
+mysql “2006-01-02 15:04:05 -0700”
+
+
+
+OPTIONS
+
+
+
+-help
+
+
+display help
+
+
+-license
+
+
+display license
+
+
+-version
+
+
+display version
+
+
+-if, -input-format
+
+
+Set format for input
+
+
+-nl, -newline
+
+
+if true add a trailing newline
+
+
+-o, -output
+
+
+output filename
+
+
+-of, -output-format
+
+
+Set format for output
+
+
+-quiet
+
+
+suppress error messages
+
+
+-utc
+
+
+timestamps in UTC
+
+
+
+EXAMPLES
+
+
+Format the date July, 7, 2016 in YYYY-MM-DD format
+