the ag_deck
+ - 0:00:00
Notes for current slide
Notes for next slide


A Tale of Two Asyncs,


or: How Language Values Shape the Features We Build



Slides, Repo Ashley Williams, StrangeLoop 2018

and the Journeys We Take to Design Them path Fig. 1: The path looks pleasant so far
1 / 166




hi, i'm ag_dubs

ashley dressed like a crab Fig. 2: Dressing like a crab brings me closer to the metal
2 / 166

nodejs

Board of Directors

3 / 166

i used to work at npm

4 / 166

nodetogether

5 / 166

invited expert

6 / 166

mozilla

7 / 166
rust
  • Core Team

  • Community Team Lead

  • crates.io Infra Lead

  • Cargo Team

  • Rust-wasm Working Group

8 / 166

rustwasm

9 / 166

rustbridge

10 / 166

rustbridge

11 / 166

rustbridge

12 / 166

rustbridge

13 / 166

I love teaching programming.

14 / 166

I love programming languages.

15 / 166

I like thinking about thinking...

16 / 166

I like thinking about thinking...

especially the type of thinking that happens when people write code.

17 / 166

graduation Fig. X: A proud, hungover, not-Computer-Science major who should not have cut her own bangs but did anyways

18 / 166

The goal of this talk is to explore two "languages" I love:

Node.js and Rust.

heart heart heart

19 / 166

Let's get philosophical...

20 / 166

Genealogy


In philosophy, genealogy is a historical technique in which one questions the commonly understood emergence of various philosophical and social beliefs by attempting to account for the scope, breadth or totality of discourse, thus extending the possibility of analysis...

Moreover, a genealogy often attempts to look beyond the discourse in question toward the conditions of its possibility.

nietzsche- the genealogy of morals Fig. 2: This mustache would like to point out you've done a bad job of questioning the origin of your values

Wikipedia, "Genealogy (philosophy)")

21 / 166


The point of a genealogical analysis is to show that a given system of thought was the result of contingent turns of history, not the outcome of rationally inevitable trends.

foucault Fig. 3: Michel Foucault is here to remind you that systems are emergent and often not a result of our conscious will

Stanford Encyclopedia of Philosophy. Michel Foucault: 3.3 From Archaeology to Geneaology

22 / 166

Geneaology is an investigation into those elements which

"we tend to feel [are] without history"

Foucault, Michel (1980). Language, Counter-Memory, Practice: Selected Essays and Interviews. Ithaca, NY: Cornell University Press. p. 139. ISBN 0-8014-9204-1.

24 / 166

Programming languages are deeply complex, historical artifacts and agents ...

25 / 166

Programming languages are deeply complex, historical artifacts and agents ...

but we rarely think of them as such.

26 / 166

thomas kuhn Fig. 7 Thomas Kuhn has trippy ideas about how ideas can and do change over time

Scientists work from models acquired through education and literature often without quite knowing or needing to know what characteristics have given these models the status of community paradigms.

Kuhn, Thomas. "The Structure of Scientific Revolutions", pg 46 (1962)

27 / 166
genealogy of programming languages Fig. 4: This diagram definitely obviously makes sense.


What is the best first programming language?

Farooq MS, Khan SA, Ahmad F, Islam S, Abid A (2014) An Evaluation Framework and Comparative Analysis of the Widely Used First Programming Languages. PLoS ONE 9(2): e88941. https://doi.org/10.1371/journal.pone.0088941

28 / 166

it's a trap Fig. 5: I wish the fools on the internet would realize this.

29 / 166

tools Fig. 6: If you are using these tools you are way closer to the metal than I am. Maybe you are too close.

"Best Tool for the Job"

30 / 166

What do programming languages want?

31 / 166

Programming Paradigms


A programming paradigm is an approach to programming a computer based on a mathematical theory or a coherent set of principles. Each paradigm supports a set of concepts that makes it the best for a certain kind of problem.

https://www.info.ucl.ac.be/~pvr/paradigmsDIAGRAMeng108.pdf

32 / 166

peter van roy Fig. 7: Computer Science is very very serious


The Paradigm Paradox


More is not better (or worse) than less, just different.

Van Roy, Peter. "Programming Paradigms for Dummies: What Every Programmer Should Know". pg 1

33 / 166

principle programming paradigms Fig. 8: Peter Van Roy's Paradigm Genealogy

https://www.info.ucl.ac.be/~pvr/paradigmsDIAGRAMeng108.pdf

34 / 166

What type of tool does Node.js want to be?

35 / 166

The way we are thinking about I/O makes things difficult.

ryan dahl Fig. 9: The font-size on that slide also makes I/O difficult.

Ryan Dahl: Original Node.js presentation (JSConf 2009). JS Conf EU.Published 8 Jun 2012.

36 / 166

Asynchronous Programming


Who can wait quietly while the mud settles? Who can remain still until the moment of action?

Laozi, Tao Te Ching


Asynchronous programming makes it possible to express waiting for long-running actions without freezing the program during these actions.

Haverbeke, Marijn. Eloquent Javascript. https://eloquentjavascript.net/11_async.html

37 / 166
var result =
db.query("select * from ?");
// use result

What is the software doing while it queries the database?

38 / 166

c10k problem

Kegel, Dan. http://www.kegel.com/c10k.html. 1999

39 / 166

Solution 1: Threads

kitten with thread Fig. 10: Kitten used thread! Ryan Dahl does not care for this one bit.

40 / 166

Threaded concurrency is a leaky abstraction. It creates pains. You have memory problems. It's hard to think about. It's not a good abstraction for what's happening on your computer.

Ryan Dahl: Original Node.js presentation (JSConf 2009). JS Conf EU.Published 8 Jun 2012.

41 / 166

nginx vs apache reqs per sec Fig. 11: There's a lot going on here, but the main takeaway is that both nginx and apache can support a whole bunch of concurrent connections.

https://blog.webfaction.com/2008/12/a-little-holiday-present-10000-reqssec-with-nginx-2/. 18 Dec 2008

42 / 166

nginx vs apache concurrency vs memory Fig. 12: Apache's memory usage grows with concurrent connections in this benchmark with dubious design.

https://blog.webfaction.com/2008/12/a-little-holiday-present-10000-reqssec-with-nginx-2/. 18 Dec 2008

43 / 166



Threads are expensive

expensive threads Fig. 13: I know, I know. This is fur and not even threads. But you get the joke. Also don't buy fur, mmkay?
44 / 166

Solution 2.1: Reactor Pattern

reactor pattern

Schmidt, Douglas C. Reactor: An Object Behavioral Pattern for Demultiplexing and Dispatching Handles for Synchronous Events. https://www.cse.wustl.edu/~schmidt/PDF/reactor-siemens.pdf

45 / 166

Solution 2.2a: libuv

reactor pattern

46 / 166

Solution 2.2b: Node.js Event Loop

reactor pattern

47 / 166

Callbacks is how I/O should work.

call me back maybe Fig. 14: I heard you like concurrency and obscure music jokes.

Ryan Dahl: Original Node.js presentation (JSConf 2009). JS Conf EU.Published 8 Jun 2012.

49 / 166

JavaScript was designed specifically to be used with an event loop. The culture of JavaScript is already geared towards evented programming.

Ryan Dahl: Original Node.js presentation (JSConf 2009). JS Conf EU.Published 8 Jun 2012.

50 / 166

Node.js wants...

To provide a purely evented, non-blocking infrastructure to script highly concurrent programs.

51 / 166

Design Goals

No function should perform direct I/O, to receive info from disk, network, or another process, there must be a callback

Low level. Stream everything. Do not remove functionality present at the POSIX layer.

Have built-in support for the most important protocols: TCP, DNS, HTTP.

Support many HTTP features.

The API should be familiar to client-side JS programmers and old school UNIX hackers. Be platform independent. Don't reinvent.

52 / 166

Evented programming is the primary paradigm for Node.js

53 / 166

What type of language does JavaScript want to be?

54 / 166

ECMAScript syntax is relaxed to enable it to serve as an easy-to-use scripting language.

https://people.mozilla.org/~jorendorff/es6-draft.html. Accessed 12 May 2015.

55 / 166

What type of language does Rust want to be?

56 / 166

Rust is a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety.

https://www.rust-lang.org/

57 / 166

Rust is a ??????? ??????????? ???????? ???? ???? ????????? ????, ???????? ?????????, ??? ????????? ????? ??????.

58 / 166

Rust was originally a systems level language for replacing C++ in Firefox.

59 / 166

sorry we're close

60 / 166

sorry we're close

61 / 166

sorry we're close

62 / 166

No run time.

No garbage collector.

No segfaults.

Static typing.

63 / 166

compiler Fig. 19: THANK U BASED COMPILER!

Baille, Lee. Illustrated Adventure Guide. RustConf 2016. https://www.slideshare.net/LizBaillie/rustconf-2016-illustrated-adventure-guide-65894363

64 / 166

Rust is a systems programming language designed to empower developers of all kinds to write fast, safe code.

66 / 166


cargo build
cargo run
cargo doc
cargo test
crates.io Fig. 15: The packages are called crates. The tool is called cargo.


First class package management

  • build tool

  • documentation generator

  • testing

  • dependency mgmt

67 / 166

cratesio Fig. 16: We call this the "pool table" design. We don't mean it as a compliment.

https://crates.io/

68 / 166

working groups

Turon, Aaron. "Announcing the 2018 Domain Working Groups!". https://internals.rust-lang.org/t/announcing-the-2018-domain-working-groups/6737. 10 February 2018.

69 / 166

Rust wants to be a first class server-side language.

70 / 166

tokio

Futures + mio = tokio

https://tokio.rs/

74 / 166

rust as node system

75 / 166

The Long Term Vision


async fn serve(addr: SocketAddr) -> io::Result<()> {
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
async for (conn, _) in socket.incoming() {
// with Tokio, read and write components are distinct:
let (reader, writer) = conn.split();
thread::spawn_task(async {
match await copy(reader, writer) {
Ok((amt, _, _)) => println!("wrote {} bytes to {}", amt, addr),
Err(e) => println!("error on {}: {}", addr, e),
};
Ok(())
});
}
Ok(())
}

Tokio Reform RFC. https://github.com/aturon/tokio-rfcs/blob/tokio-reform/tokio-reform.md

76 / 166

Rust wants to be a first class server side language.

77 / 166

Rust wants to be a first class server side language.

But should it?

78 / 166

big question

Should you write a web service with a systems level language?

79 / 166

Should you write a webservice with a language designed to write a browser?

80 / 166

firefox nightly

Rust was originally designed to replace C++ in Firefox

81 / 166

servo

Rust was originally designed to replace C++ in Firefox

82 / 166

stylo

Firefox Quantum shipped with a first-of-kind parallelized Style System, Stylo, last year.

Matthews, Josh. The Story of Stylo. Rust Belt Rust Conference, 2017. slides, video

83 / 166

program workingness

Evans, Julia. "Learning systems programming with Rust". 11 September 2016. https://jvns.ca/blog/2016/09/11/rustconf-keynote/.

84 / 166

possible

Evans, Julia. "Learning systems programming with Rust". 11 September 2016. https://jvns.ca/blog/2016/09/11/rustconf-keynote/.

85 / 166

empowering

Evans, Julia. "Learning systems programming with Rust". 11 September 2016. https://jvns.ca/blog/2016/09/11/rustconf-keynote/.

86 / 166

systems programmers

Evans, Julia. "Learning systems programming with Rust". 11 September 2016. https://jvns.ca/blog/2016/09/11/rustconf-keynote/.

87 / 166

Should you write a webservice with a language designed to write a browser?

88 / 166

Should you write a webservice with a language designed to write a browser?

Yes.

89 / 166

Concurrency vs. Memory

90 / 166

Speed, Safety, Productivity:

Pick 3

91 / 166

⚠️ Productivity ⚠️

92 / 166

developer experience

Developer Experience

93 / 166

developer experience

Developer Experience

94 / 166

developer experience

Developer Experience

95 / 166

The history of every major galactic civilization tends to pass through three distinct and recognizable phases, those of Survival, Inquiry and Sophistication, otherwise known as the How, Why, and Where phases. For instance, the first phase is characterized by the question ‘How can we eat?’, the second by the question ‘Why do we eat?’ and the third by the question, ‘Where shall we have lunch?’

Douglas Adams

The Hitchhiker's Guide To The Galaxy

Tao, Terry. "There’s more to mathematics than rigour and proofs". https://terrytao.wordpress.com/career-advice/theres-more-to-mathematics-than-rigour-and-proofs/

96 / 166

Stages of Learning

  • Pre-Rigorous (Survival) - blindly apply formalism, very intuitive

  • Rigorous (Inquiry) - intense "proper" application of formalism

  • Post-Rigorous (Sophistication) - a return to intuition, with the support of deeply understood formalism

Tao, Terry. "There’s more to mathematics than rigour and proofs". https://terrytao.wordpress.com/career-advice/theres-more-to-mathematics-than-rigour-and-proofs/

97 / 166

Stroustrup's Rule

  • For new features, people insist on LOUD explicit syntax.

  • For established features, people want terse notation.

Herman, Dave. "Stroustrup's Rule and Layering Over Time". https://thefeedbackloop.xyz/stroustrups-rule-and-layering-over-time/. 15 December 2016.

98 / 166
let file = match File::open("file.txt") {
Ok(file) => file,
Err(err) => { return err; }
}
let file = try!(File::open("file.txt"));
let file = File::open("file.txt")?;
99 / 166

The Mechanism is not the Model

I see this a lot in standards and design: as we spend more and more of our time swimming in formalisms like specs, semantics, or implementations, it's easy for us to overlook those "fuzzier intuitions." Just like in mathematics, rigorous tools like specs help us to be precise, but the mechanism of our designs and the developer's mental model they support are two separate things.

Herman, Dave. "The Mechanism is not the Mental Model". https://thefeedbackloop.xyz/the-mechanism-is-not-the-mental-model/. 01 December 2016.

100 / 166

Rust tends to build features for the Survival and Sophistication stages.

101 / 166

Rust tends to build features for the Survival and Sophistication stages.

The folks in the Inquiry stage find this incredibly frustrating.

102 / 166

We don't have all the answers.

103 / 166

We don't have all the answers.

But we are trying our best to ask the right questions.

104 / 166

Can Rust achieve the Ergonomics and Productivity required to make systems level control viable for Web Development?

105 / 166

oxidizing source maps

Fitzgerald, Nick. "Oxidizing Source Maps with Rust and WebAssembly". https://hacks.mozilla.org/2018/01/oxidizing-source-maps-with-rust-and-webassembly/ 18 January 2018.

107 / 166

scalajssource

Fitzgerald, Nick. "Oxidizing Source Maps with Rust and WebAssembly". https://hacks.mozilla.org/2018/01/oxidizing-source-maps-with-rust-and-webassembly/ 18 January 2018.

108 / 166

maybe you dont need rust and wasm

109 / 166

v8

110 / 166

Speed Without Wizardry


The “Oxidizing” article recounts my experience integrating Rust (compiled to WebAssembly) into the source-map JavaScript library. Although the JavaScript implementation was originally authored in idiomatic JavaScript style, as we profiled and implemented speed improvements, the code became hard to read and maintain. With Rust and its zero-cost abstractions, we found that there was no trade-off between performance and clean code.

Fitzgerald, Nick. "Speed Without Wizardry". http://fitzgeraldnick.com/2018/02/26/speed-without-wizardry.html

111 / 166

How did Rust get here?

112 / 166

rust oss

Turon, Aaron and Niko Matsakis. "Why wasn't I consulted? How Rust does OSS". Mozilla SF All Hands 2018. https://docs.google.com/presentation/d/1Cz0jGrAfFZVr-31aG586bnSZpCQusRvca93NleTI0l0/

113 / 166

“Why wasn't I consulted” is the fundamental question of the web. It is the rule from which other rules are derived. Humans have a fundamental need to be consulted, engaged, to exercise their knowledge (and thus power), and no other medium that came before has been able to tap into that as effectively.

Paul Ford

Ford, Paul. "The Web Is A Customer Service Platform." http://www.ftrain.com/wwic.html

115 / 166

"Consultation", Rust-style


  • All “major” decisions are preceded by an RFC

  • Goal: catalog tradeoffs and explore designs

  • Team drives to steady state and consensus

  • No New Rationale: decisions rest on prior, public debate

Turon, Aaron and Niko Matsakis. "Why wasn't I consulted? How Rust does OSS". Mozilla SF All Hands 2018. https://docs.google.com/presentation/d/1Cz0jGrAfFZVr-31aG586bnSZpCQusRvca93NleTI0l0/

116 / 166

Lazy Consensus means that when you are convinced that you know what the community would like to see happen you can simply assume that you already have consensus and get on with the work.

Apache Software Foundation

Lazy Consensus. https://rave.apache.org/docs/governance/lazyConsensus.html

117 / 166

Articulate the Vision


  • “Thought Leadership”: leaders need to expose a way of thinking

  • Inspire the community toward a clear goal

  • Establish shared values

  • Identify and explain historical themes

Turon, Aaron and Niko Matsakis. "Why wasn't I consulted? How Rust does OSS". Mozilla SF All Hands 2018. https://docs.google.com/presentation/d/1Cz0jGrAfFZVr-31aG586bnSZpCQusRvca93NleTI0l0/

118 / 166

Every good work of software starts by scratching a developer's personal itch.

“The Cathedral and the Bazaar”

Not gonna cite this one because the dude that wrote this is a big jerk.

119 / 166

Rust's Roadmap Process


  • Vision and values happen at many scales

  • Roadmap helps people “itch” in the same way

  • … and also makes it easier to say “no”

  • Rust uses RFCs even for its roadmap

Turon, Aaron and Niko Matsakis. "Why wasn't I consulted? How Rust does OSS". Mozilla SF All Hands 2018. https://docs.google.com/presentation/d/1Cz0jGrAfFZVr-31aG586bnSZpCQusRvca93NleTI0l0/

120 / 166

Problems of Scale


  • RFC threads can get 100 civil, insightful comments in 24 hours

  • Information overload: there’s so much out there that no one can find anything

  • Never enough managers!

Turon, Aaron and Niko Matsakis. "Why wasn't I consulted? How Rust does OSS". Mozilla SF All Hands 2018. https://docs.google.com/presentation/d/1Cz0jGrAfFZVr-31aG586bnSZpCQusRvca93NleTI0l0/

124 / 166

The Role of Pluralism


  • Not everyone will share all values; that’s OK

  • Not everyone will interpret values the same way; that’s OK too

  • Making room for a diversity of values makes for a better product

  • Project leaders must turn this input into a singular, coherent vision

Turon, Aaron and Niko Matsakis. "Why wasn't I consulted? How Rust does OSS". Mozilla SF All Hands 2018. https://docs.google.com/presentation/d/1Cz0jGrAfFZVr-31aG586bnSZpCQusRvca93NleTI0l0/

125 / 166

What's the point of all of this?

126 / 166

I don't think that Rust, especially its async story, would be the same if it were not for Node

127 / 166

Node.js exists for a very specific reason, and is a response to the historical and material situation it was written in.

128 / 166

Node.js exists for a very specific reason, and is a response to the historical and material situation it was written in.

Rust, is, at least, partially a response to Node.

129 / 166

However...

130 / 166

I think there are ways that Node.js, and JavaScript, can also respond to Rust.

131 / 166

Node.js and JavaScript have not rearticulated their goals and audience in a long time.

132 / 166

Node.js and JavaScript have not rearticulated their goals and audience in a long time.

Rust also has room to improve here.

133 / 166

As a result, I think there is a lack of overall vision.

134 / 166

As a result, I think there is a lack of overall vision.

We are working extremely hard- but to what end? and for who?

135 / 166

A lack of vision causes, at best, a mismatch in expectations.

136 / 166

not explicit

Boats, Without. "Not Explicit". https://boats.gitlab.io/blog/post/2017-12-27-things-explicit-is-not/. 27 December 2017.

137 / 166

A lack of vision can also have much, much worse implications.

138 / 166

bjarne stroustrup

Bjarne Stroustrup

Creator of C++

https://en.wikiquote.org/wiki/File:BjarneStroustrup.jpg

139 / 166

Many/most people in WG21 are working independently towards non-shared goals. Individually, many (most?) proposals make sense. Together they are insanity to the point of endangering the future of C++.

Stroustrup, Bjarne. "Remember the Vasa!" http://open-std.org/JTC1/SC22/WG21/docs/papers/2018/p0977r0.pdf. 6 March 2018.

140 / 166

Remember the Vasa!

vasa

141 / 166

The Vasa (or Wasa) was a Swedish galleon , one of the largest and most heavily armed warships of its time. Already at the beginning of its maiden voyage on 10 August 1628, the Vasa sank after only about 1300 meters driving distance in normal swell due to serious structural instability.

https://de.wikipedia.org/wiki/Vasa_(Schiff)

142 / 166

We need a reasonably coherent language that can be used by “ordinary programmers” whose main concern is to ship great applications on time. We now have about 150 cooks; that’s not a good way to get a tasty and balanced meal.

We are on the path to something that could destroy C++. We must get off that path!

Stroustrup, Bjarne. "Remember the Vasa!" http://open-std.org/JTC1/SC22/WG21/docs/papers/2018/p0977r0.pdf. 6 March 2018.

143 / 166

Ergonomics and Productivity are the determinants by which the success of any programming language will be measured.

145 / 166

In order to get those right, we need our users to be in the room.

146 / 166

To make Go successful we need everyone's help, and everyone isn't here.

Russ Cox

"Keynote: Go, Open Source, Community" Gophercon 2015. https://blog.golang.org/open-source

147 / 166

The major problem caused by the unavailability of feature uniformity is that the size of the language increases, which in turn, results into a longer learning curve.

Farooq MS, Khan SA, Ahmad F, Islam S, Abid A (2014) An Evaluation Framework and Comparative Analysis of the Widely Used First Programming Languages. PLoS ONE 9(2): e88941. https://doi.org/10.1371/journal.pone.0088941

148 / 166

This leads to an increase in the language complexity, in terms of the number and type of constructs. Resultantly, a language without feature uniformity usually demands the instructor to teach a subset of the language. A programmer can use this subset for writing code, but for reading other's code, comprehensive knowledge is required.

Farooq MS, Khan SA, Ahmad F, Islam S, Abid A (2014) An Evaluation Framework and Comparative Analysis of the Widely Used First Programming Languages. PLoS ONE 9(2): e88941. https://doi.org/10.1371/journal.pone.0088941

149 / 166

I'm a little sad that maybe JavaScript is becoming not a great First Programming Language

150 / 166

I'm a little sad that maybe JavaScript is becoming not a great First Programming Language

But moreso, I'm genuinely sad that if that is true, it likely happened by accident

151 / 166

ECMAScript syntax is relaxed to enable it to serve as an easy-to-use scripting language.

https://people.mozilla.org/~jorendorff/es6-draft.html. Accessed 12 May 2015.

152 / 166

javascript

Williams, Ashley. "If you wish to learn ES6 from scratch you must first invent the universe". https://github.com/ashleygwilliams/jsconf-2015-deck. July 2015.

153 / 166

I'd like to see all language communities better articulate their visions.

154 / 166

I'd like to see more (requests for) feedback, from a wide range of people.

155 / 166

I'd like to see a greater more visible realization that the artifacts we are build are historical artifacts.

156 / 166

I'd like to see a greater more visible realization that the artifacts we are build are historical artifacts.

Products of their time. A time that is certainly going to change.

157 / 166

This is not always fun.

158 / 166

This is definitely uncomfortable.

159 / 166

The technology we build is NOT the product of rationally inevitable trends.

160 / 166

The technology we build is NOT the product of rationally inevitable trends.

It's the result of contingent turns of history.

161 / 166

History is going to move forward.

162 / 166

History is going to move forward.

We need to be prioritizing the work of understanding that history, and we are not.

163 / 166

It is the work to be done!

164 / 166

And I'm excited to do it.

I hope more folks will join us.

165 / 166


airtrain Fig. 18: Looking back on the Air Train at the Munich Airport.
166 / 166




hi, i'm ag_dubs

ashley dressed like a crab Fig. 2: Dressing like a crab brings me closer to the metal
2 / 166
Paused

Help

Keyboard shortcuts

, , Pg Up, k Go to previous slide
, , Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
t Restart the presentation timer
?, h Toggle this help
Esc Back to slideshow