3.7 KiB
3.7 KiB
Lesson 04 | Data Wrangling
Links
- TOC | Missing Semester
- Playlist: Missing Semester
- class notes
- Curr: https://youtu.be/sz_dsktIjt4?si=oVhYD6bPPM6FzWSL&t=1325
timestamps
- 00:00 - introduction
- 06:55 - Stream Editor
- 07:36 - Replacement Expressions
- 08:58 - Regular Expression
- 09:20 - Regular Expressions
- 10:20 - Square Brackets
- 11:33 - Add Modifiers
- 12:56 - Alternations
- 17:09 - Anchoring the Regular Expression
- 18:58 - Capture Groups
- 20:15 - Regular Expression Debugger current
- 24:10 - Regular Sessions
- 26:01 - Match and Email Address
- 29:03 - Sort
- 34:00 - Awk
- 38:44 - Berkeley Calculator
- 40:37 - Computer Statistics over Inputs
- 41:22 - Summary Statistics
- 41:53 - Plotting
- 42:50 - Two sort of special types
- 45:54 - example where data wrangling is useful
- 46:45 - image captures to standard output
- 47:26 - operate on standard input
- 48:00 - display in an image display
notes
intro example
- using ssh someserver 'somecommand' will run that command on the server
-
you could run a series of commands on the server instead of channeling all that info back
ssh someserver 'journalctl | grep sshd | grep "Disconnected from"' | less - this will run journalctl on the server, find anything that says 'sshd' and 'disconnec..' in the results
- then send all those results back to our machine where we pipe it through 'less'
SED
- stream editor
- allows you to make changes to the contents of a stream
- full programming langauge
- common task is to run replacement expressions on an input stream
example
sed 's/.*blahblah blah//'-
uses regular expressions
- way of matching text
sed modifiers
- (ab)* - remove zero or more of 'ab'
- -E use new replacement
- (ab|bc)* - remove 'ab' or 'bc'
regex debugger
- regex101.com