Rust LDN 24 7 19 Oxidising the Command Line

Oxidising the Command Line
July 24. 2019
mattpro@yelp.com

Rust at Yelp Yelp connects Users with Great Local Businesses
184 million reviews
35 million unique devices per month
Yelp has developed a framework in Rust for real-time
A/B testing. It’s used across all Yelp websites and
apps, and experiment subjects range from UX to
internal infrastructure. Rust was chosen because it’s as
fast as C (cheap to run) and safer than C (cheap to
maintain).
5,550 employees worldwide

Oxidising the
Command Line
Basic toolbox for writing CLI utilities:
Handling Broken Pipes
Reading input by byte
Reading lines
Reading input from stdin or ﬁles
Handling non-UTF8 arguments

Oxidising the
Command Line
Handling Broken Pipes

fn main() {
match std::env::args().nth(1) {
Some(y) => loop {
println!("{}", y)
}
None => loop {
println!("y")
}
}
}

% target/debug/yes | /usr/bin/head
y
y
y
y
y
y
y
y
y
y
thread 'main' panicked at 'failed printing to stdout: Broken pipe (os error 32)',
libstd/io/stdio.rs:700:9
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
% echo "${pipestatus[1]}"
101

% /usr/bin/yes | /usr/bin/head
y
y
y
y
y
y
y
y
y
y
141

#define SIGHUP 1 /* hangup */
#define SIGINT 2 /* interrupt */
#define SIGQUIT 3 /* quit */
#define SIGILL 4 /* illegal instruction (not reset when caught) */
#define SIGTRAP 5 /* trace trap (not reset when caught) */
#define SIGABRT 6 /* abort() */
#define SIGFPE 8 /* floating point exception */
#define SIGKILL 9 /* kill (cannot be caught or ignored) */
#define SIGBUS 10 /* bus error */
#define SIGSEGV 11 /* segmentation violation */
#define SIGSYS 12 /* bad argument to system call */
#define SIGPIPE 13 /* write on a pipe with no one to read it */
#define SIGALRM 14 /* alarm clock */
#define SIGTERM 15 /* software termination signal from kill */
#define SIGURG 16 /* urgent condition on IO channel */
#define SIGSTOP 17 /* sendable stop signal not from tty */
#define SIGTSTP 18 /* stop signal from tty */
#define SIGCONT 19 /* continue a stopped process */
#define SIGCHLD 20 /* to parent on child stop or exit */
...

use std::io::{stdout, Write};
fn main() {
let mut out = stdout();
match std::env::args().nth(1) {
Some(y) => loop {
writeln!(out, "{}", y).unwrap()
}
None => loop {
writeln!(out, "y").unwrap()
}
}
}

y
y
y
y
y
y
y
y
y
y
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 32,
kind: BrokenPipe, message: "Broken pipe" }', libcore/result.rs:1009:5

use std::io::{stdout, ErrorKind::BrokenPipe, Write};
fn main() {
let arg = std::env::args().nth(1);
let y = match arg.as_ref() {
Some(a) => a,
None => "y",
};
loop {
if let Err(e) = writeln!(out, "{}", y) {
if e.kind() == BrokenPipe {
std::process::exit(141)
} else {
panic!("{:?}", e)
}
}
}
}

y
y
y
y
y
y
y
y
y
y
141

Oxidising the
Command Line
Reading input by byte

Trait std::io::Read
fn bytes(self) -> Bytes<Self>
Transforms this Read instance to an Iterator over its bytes.
The returned type implements Iterator where the Item is Result<u8,io::Error>. The yielded item is Ok if a
byte was successfully read and Err otherwise. EOF is mapped to returning None from this iterator.

use std::io::{stdin, stdout, Read, Write};
fn main() {
let fp = stdin().bytes();
let mut cnt = 10;
for ch in fp {
if let Ok(b) = ch {
out.write(b).unwrap();
if b == b'n' {
cnt -= 1
}
} else { // Err
cnt -= 1
}
if cnt == 0 {
break
}
}
}

error[E0308]: mismatched types
--> src/bin/head.rs:13:23
|
13 | out.write(b).unwrap();
| ^ expected &[u8], found u8
|
= note: expected type `&[u8]`
found type `u8`

Trait std::io::Write
fn write(&mut self, buf: &[u8]) -> Result<usize>

Module std::slice
pub fn from_ref<T>(s: &T) -> &[T]
Converts a reference to T into a slice of length 1 (without copying).

use std::io::{stdin, stdout, Read, Write};
fn main() {
let fp = stdin().bytes();
let mut cnt = 10;
for ch in fp {
if let Ok(b) = ch {
out.write(std::slice::from_ref(&b)).unwrap();
if b == b'n' {
cnt -= 1
}
} else { // Err
cnt -= 1
}
if cnt == 0 {
break
}
}
}

Oxidising the
Command Line
Reading lines

Trait std::io::BufRead
fn read_line(&mut self, buf: &mut String) -> Result<usize>
Read all bytes until a newline (the 0xA byte) is reached, and append them to the provided buffer.
This function will read bytes from the underlying stream until the newline delimiter (the 0xA byte) or EOF is found.
Once found, all bytes up to, and including, the delimiter (if found) will be appended to buf.
If successful, this function will return the total number of bytes read.
If this function returns Ok(0), the stream has reached EOF.

fn lines(self) -> Lines<Self>
Returns an iterator over the lines of this reader.
The iterator returned from this function will yield instances ofio::Result<String>. Each string returned will not
have a newline byte (the 0xA byte) or CRLF (0xD, 0xA bytes) at the end.

fn read_until(&mut self, byte: u8, buf: &mut Vec<u8>) -> Result<usize>
Read all bytes into buf until the delimiter byte or EOF is reached.
This function will read bytes from the underlying stream until the delimiter or EOF is found. Once found, all bytes up
to, and including, the delimiter (if found) will be appended to buf.
If successful, this function will return the total number of bytes read.

use std::io::{stdin, BufRead};
fn main() {
let si = stdin();
let mut stream = si.lock();
let mut line = Vec::new();
let mut linect = 0;
loop {
match stream.read_until(b'n', &mut line) {
Ok(0) => break, // EOF
Ok(_) => linect += 1,
Err(_) => break
}
line.clear()
}
println!("{}", linect);
}

Oxidising the
Command Line
Reading input from stdin or ﬁles

void cook_args(char **argv) {
FILE *fp;
fp = stdin;
do {
if (*argv) {
if (!strcmp(*argv, "-"))
fp = stdin;
else if ((fp = fopen(*argv, "r")) == NULL) {
warn("%s", *argv);
rval = 1;
++argv;
continue;
}
}
cook_buf(fp);
if (fp == stdin)
clearerr(fp);
else
(void)fclose(fp);
} while (*argv);
}

Struct std::io::Stdin
impl Read for Stdin
Struct std::fs::File
impl Read for File

use std::io::{stdin, BufRead, BufReader};
let mut fp: Box<BufRead>;
let si = stdin();
let args: Vec<_> = std::env::args().collect();
let mut argv = &args[1..]; // skip the binary name at args[0]
loop {
if argv.is_empty() {
fp = Box::new(si.lock());
} else {
if argv[0] == "-" {
fp = Box::new(si.lock());
} else {
fp = Box::new(BufReader::new(File::open(&argv[0]).unwrap()));
}
argv = &argv[1..];
}
do_something(fp);
if argv.is_empty() {
break
}
}

Oxidising the
Command Line
Handling non-UTF8 arguments

let args: Vec<_> = std::env::args().collect();

% target/debug/cat rutt^I$'266'$'303'k
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: "ruttxB6xC3k"',
src/libcore/result.rs:999:5
% echo $?
101

% target/debug/cat rutt^I$'266'$'303'k
% echo $?
101
% env RUST_BACKTRACE=1 target/debug/cat rutt$'266'$'303'k
stack backtrace:
...
14: core::iter::traits::iterator::Iterator::collect
at
/rustc/08bfe16129b0621bc90184f8704523d4929695ef/src/libcore/iter/traits/iterator.rs:1465
15: cat::main
at src/bin/cat.rs:6
...

#6 0x000055555556c53a in unwrap<alloc::string::String,std::ffi::os_str::OsString> ()
at /rustc/08bfe16129b0621bc90184f8704523d4929695ef/src/libcore/result.rs:800
800 Err(e) => unwrap_failed("called `Result::unwrap()` on an `Err` value", e),
(gdb) up
#7 {{closure}} () at src/libstd/env.rs:746
746 self.inner.next().map(|s| s.into_string().unwrap())
(gdb) list
741
742 #[stable(feature = "env", since = "1.0.0")]
743 impl Iterator for Args {
744 type Item = String;
745 fn next(&mut self) -> Option<String> {
746 self.inner.next().map(|s| s.into_string().unwrap())
747 }
748 fn size_hint(&self) -> (usize, Option<usize>) { self.inner.size_hint() }
749 }

let argv: Vec<_> = std::env::args_os().collect();

Struct getopts::Matches
pub struct Matches {
pub free: Vec<String>,
// some fields omitted
}
The result of checking command line arguments. Contains a vector of matches and a vector of free strings.
Fields
free: Vec<String>
Free string fragments

Struct clap::ArgMatches
pub fn values_of_os<S: AsRef<str>>(&'a self, name: S) -> Option<OsValues<'a>>
Gets a OsValues struct which is implements Iterator for OsString values of a speciﬁc argument. If the option
wasn't present at runtime it returns None. An OS value on Unix-like systems is any series of bytes, regardless of whether
or not they contain valid UTF-8 code points. Since Strings in Rust are guaranteed to be valid UTF-8, a valid ﬁlename as
an argument value on Linux (for example) may contain invalid UTF-8 code points.

use clap::{Arg, App};
fn main() {
let matches = App::new("cat")
.arg(Arg::with_name("sflag")
.short("s"))
.arg(Arg::with_name("FILE")
.multiple(true))
.get_matches();
let args: Vec<_> = matches.values_of_os("FILE")
.unwrap()
.collect();
std::process::exit(raw_args(&args));
}
fn raw_args(mut argv: &[&OsStr]) -> i32 {
...
}

Questions/Suggestions?
Please contact me at mattpro@yelp.com

We’re Hiring in London!
https://www.yelp.com/careers

Rust LDN 24 7 19 Oxidising the Command Line

More Related Content

What's hot

Similar to Rust LDN 24 7 19 Oxidising the Command Line

Recently uploaded

Rust LDN 24 7 19 Oxidising the Command Line