What Cloud Engineers Could Learn from Clouds

A presentation at KCDC 2023 in June 2023 in Kansas City, MO, USA by Laura Santamaria

Slide 1

Slide 1

What Cloud Engineers Could Learn from Clouds Laura Santamaria, Lead Developer Advocate Dell Technologies @nimbinatus (hachyderm)

Slide 2

Slide 2

@nimbinatus (hachyderm) Not these Photo by Taylor Vick on Unsplash 2 Copyright © Dell Inc. All Rights Reserved.

Slide 3

Slide 3

@nimbinatus (hachyderm) These Own work 3 Copyright © Dell Inc. All Rights Reserved.

Slide 4

Slide 4

“WAIT, AREN’T YOU A DEVELOPER ADVOCATE?” Intro @nimbinatus (hachyderm) 4 Copyright © Dell Inc. All Rights Reserved.

Slide 5

Slide 5

@nimbinatus (hachyderm) History 5 Copyright © Dell Inc. All Rights Reserved.

Slide 6

Slide 6

@nimbinatus (hachyderm) Planetary Atmospheres Credits: NASA/JPL-Caltech/SwRI/MSSS/ Gerald Eichstädt /Seán Doran © CC NC SA 6 Copyright © Dell Inc. All Rights Reserved.

Slide 7

Slide 7

@nimbinatus (hachyderm) Clouds and Clouds 7 Copyright © Dell Inc. All Rights Reserved.

Slide 8

Slide 8

Definitions @nimbinatus (hachyderm) 8 Copyright © Dell Inc. All Rights Reserved.

Slide 9

Slide 9

Complex Systems @nimbinatus (hachyderm) 9 Copyright © Dell Inc. All Rights Reserved.

Slide 10

Slide 10

@nimbinatus (hachyderm) A cloud physics example “Hoh Rain Forest Entrance Sign” by Adbar is licensed under CC BY-SA 3.0 10 Copyright © Dell Inc. All Rights Reserved.

Slide 11

Slide 11

@nimbinatus (hachyderm) A cloud computing example Photo by Jordan Harrison on Unsplash 11 Copyright © Dell Inc. All Rights Reserved.

Slide 12

Slide 12

Chaos and Chaotic Systems @nimbinatus (hachyderm) 12 Copyright © Dell Inc. All Rights Reserved.

Slide 13

Slide 13

@nimbinatus (hachyderm) A cloud physics example Photo by 2 Bro’s Media on Unsplash 13 Copyright © Dell Inc. All Rights Reserved.

Slide 14

Slide 14

@nimbinatus (hachyderm) A cloud computing example Photo by Jon Tyson on Unsplash 14 Copyright © Dell Inc. All Rights Reserved.

Slide 15

Slide 15

Patterns @nimbinatus (hachyderm) 15 Copyright © Dell Inc. All Rights Reserved.

Slide 16

Slide 16

@nimbinatus (hachyderm) A cloud physics example Photo by Jordan Steranka on Unsplash 16 Copyright © Dell Inc. All Rights Reserved.

Slide 17

Slide 17

@nimbinatus (hachyderm) A cloud computing example Photo by Brian McGowan on Unsplash 17 Copyright © Dell Inc. All Rights Reserved.

Slide 18

Slide 18

@nimbinatus (hachyderm) All of this is what modeling is about. 18 Copyright © Dell Inc. All Rights Reserved.

Slide 19

Slide 19

Learning from clouds @nimbinatus (hachyderm) 19 Copyright © Dell Inc. All Rights Reserved.

Slide 20

Slide 20

OBSERVATIONS AND DATA Understanding chaotic systems @nimbinatus (hachyderm) 20 Copyright © Dell Inc. All Rights Reserved.

Slide 21

Slide 21

@nimbinatus (hachyderm) Observations Photo by Daniel Lerman on Unsplash 21 Copyright © Dell Inc. All Rights Reserved.

Slide 22

Slide 22

@nimbinatus (hachyderm) Experiments Photo by CHUTTERSNAP on Unsplash 22 Copyright © Dell Inc. All Rights Reserved.

Slide 23

Slide 23

@nimbinatus (hachyderm) Observation and Experimentation Build Destroy Repeat Mindful Experimentation 23 Copyright © Dell Inc. All Rights Reserved. Chaos Experiments

Slide 24

Slide 24

HOW WE MAKE SENSE OF THINGS Modeling and predicting behavior @nimbinatus (hachyderm) 24 Copyright © Dell Inc. All Rights Reserved.

Slide 25

Slide 25

@nimbinatus (hachyderm) Data in, data out Photo by NOAA on Unsplash 25 Copyright © Dell Inc. All Rights Reserved.

Slide 26

Slide 26

@nimbinatus (hachyderm) Scale matters Photo by NOAA on Unsplash 26 Copyright © Dell Inc. All Rights Reserved.

Slide 27

Slide 27

@nimbinatus (hachyderm) Modeling in computing systems UML TLA+ 27 Copyright © Dell Inc. All Rights Reserved. Alloy

Slide 28

Slide 28

@nimbinatus (hachyderm) TLA+ example (truncated) From https:// www.hillelwayne.com/ modeling-deployments/ … variables servers = {“s1”, “s2”, “s3”}, load_balancer = servers, update_flag = [s \in servers |-> FALSE], updated = [s \in servers |-> FALSE]; define SameVersion == \A x, y \in load_balancer: updated[x] = updated[y] ZeroDowntime == \E server \in load_balancer: updated[server] /= UPDATING end define fair process update_server \in servers begin Update: await update_flag[self]; updated[self] := UPDATING; FinishUpdate: 28 Copyright © Dell Inc. All Rights Reserved. updated[self] := TRUE; end process; fair process start_update = “start_update” begin StartUpdate: load_balancer := {“s1”}; update_flag := [s \in servers |-> IF s = “s1” THEN FALSE ELSE TRUE]; Transition: await \A s \in (servers \ load_balancer): updated[s] = TRUE; load_balancer := servers \ load_balancer; update_flag[“s1”] := TRUE; Finish: await updated[“s1”] = TRUE; load_balancer := load_balancer \union {“s1”}; end process; …

Slide 29

Slide 29

@nimbinatus (hachyderm) TLA+ example (truncated) From https:// www.hillelwayne.com/ modeling-deployments/ … variables servers = {“s1”, “s2”, “s3”}, load_balancer = servers, update_flag = [s \in servers |-> FALSE], updated = [s \in servers |-> FALSE]; define SameVersion == \A x, y \in load_balancer: updated[x] = updated[y] ZeroDowntime == \E server \in load_balancer: updated[server] /= UPDATING end define fair process update_server \in servers begin Update: await update_flag[self]; updated[self] := UPDATING; FinishUpdate: updated[self] := TRUE; end process; fair process start_update = “start_update” begin StartUpdate: load_balancer := {“s1”}; update_flag := [s \in servers |-> IF s = “s1” THEN FALSE ELSE TRUE]; Transition: await \A s \in (servers \ load_balancer): updated[s] = TRUE; load_balancer := servers \ load_balancer; update_flag[“s1”] := TRUE; Finish: await updated[“s1”] = TRUE; load_balancer := load_balancer \union {“s1”}; Copyright © Dell Inc. All Rights Reserved. 29 end process;

Slide 30

Slide 30

@nimbinatus (hachyderm) TLA+ example (truncated) From https:// www.hillelwayne.com/ modeling-deployments/ … variables servers = {“s1”, “s2”, “s3”}, load_balancer = servers, update_flag = [s \in servers |-> FALSE], updated = [s \in servers |-> FALSE]; define SameVersion == \A x, y \in load_balancer: updated[x] = updated[y] ZeroDowntime == \E server \in load_balancer: updated[server] /= UPDATING end define fair process update_server \in servers begin Update: await update_flag[self]; updated[self] := UPDATING; FinishUpdate: updated[self] := TRUE; end process; fair process start_update = “start_update” begin StartUpdate: load_balancer := {“s1”}; update_flag := [s \in servers |-> IF s = “s1” THEN FALSE ELSE TRUE]; Transition: await \A s \in (servers \ load_balancer): Copyright © Dell Inc. All Rights Reserved. 30 updated[s] = TRUE;

Slide 31

Slide 31

@nimbinatus (hachyderm) TLA+ example (truncated) From https:// www.hillelwayne.com/ modeling-deployments/ servers = {“s1”, “s2”, “s3”}, load_balancer = servers, update_flag = [s \in servers |-> FALSE], updated = [s \in servers |-> FALSE]; define SameVersion == \A x, y \in load_balancer: updated[x] = updated[y] ZeroDowntime == \E server \in load_balancer: updated[server] /= UPDATING end define fair process update_server \in servers begin Update: await update_flag[self]; updated[self] := UPDATING; FinishUpdate: updated[self] := TRUE; end process; fair process start_update = “start_update” begin StartUpdate: load_balancer := {“s1”}; update_flag := [s \in servers |-> IF s = “s1” THEN FALSE ELSE TRUE]; Transition: await \A s \in (servers \ load_balancer): updated[s] = TRUE; load_balancer := servers \ load_balancer; update_flag[“s1”] := TRUE; Finish: await updated[“s1”] = TRUE; load_balancer := load_balancer \union {“s1”}; end process; Copyright © Dell Inc. All Rights Reserved. 31 …

Slide 32

Slide 32

@nimbinatus (hachyderm) TLA+ example (truncated) From https:// www.hillelwayne.com/ modeling-deployments/ … variables servers = {“s1”, “s2”, “s3”}, load_balancer = servers, update_flag = [s \in servers |-> FALSE], updated = [s \in servers |-> FALSE]; define SameVersion == \A x, y \in load_balancer: updated[x] = updated[y] ZeroDowntime == \E server \in load_balancer: updated[server] /= UPDATING end define fair process update_server \in servers begin Update: await update_flag[self]; updated[self] := UPDATING; FinishUpdate: updated[self] := TRUE; end process; fair process start_update = “start_update” begin StartUpdate: load_balancer := {“s1”}; update_flag := [s \in servers |-> IF s = “s1” THEN FALSE ELSE TRUE]; Transition: await \A s \in (servers \ load_balancer): updated[s] = TRUE; load_balancer := servers \ load_balancer; update_flag[“s1”] := TRUE; Finish: await updated[“s1”] = TRUE; load_balancer := load_balancer \union {“s1”}; end process; … 32 Copyright © Dell Inc. All Rights Reserved.

Slide 33

Slide 33

@nimbinatus (hachyderm) TLA+ example (truncated) From https:// www.hillelwayne.com/ modeling-deployments/ variables servers = {“s1”, “s2”, “s3”}, load_balancer = servers, update_flag = [s \in servers |-> FALSE], updated = [s \in servers |-> FALSE]; define SameVersion == \A x, y \in load_balancer: updated[x] = updated[y] ZeroDowntime == \E server \in load_balancer: updated[server] /= UPDATING end define fair process update_server \in servers begin Update: await update_flag[self]; updated[self] := UPDATING; FinishUpdate: updated[self] := TRUE; end process; fair process start_update = “start_update” begin StartUpdate: load_balancer := {“s1”}; update_flag := [s \in servers |-> IF s = “s1” THEN FALSE ELSE TRUE]; Transition: await \A s \in (servers \ load_balancer): updated[s] = TRUE; load_balancer := servers \ load_balancer; update_flag[“s1”] := TRUE; Finish: await updated[“s1”] = TRUE; load_balancer := load_balancer \union {“s1”}; end process; … Copyright © Dell Inc. All Rights Reserved. 33

Slide 34

Slide 34

@nimbinatus (hachyderm) TLA+ example (truncated) From https:// www.hillelwayne.com/ modeling-deployments/ … variables servers = {“s1”, “s2”, “s3”}, load_balancer = servers, update_flag = [s \in servers |-> FALSE], updated = [s \in servers |-> FALSE]; define SameVersion == \A x, y \in load_balancer: updated[x] = updated[y] ZeroDowntime == \E server \in load_balancer: updated[server] /= UPDATING end define fair process update_server \in servers begin Update: await update_flag[self]; updated[self] := UPDATING; FinishUpdate: updated[self] := TRUE; end process; fair process start_update = “start_update” begin StartUpdate: load_balancer := {“s1”}; update_flag := [s \in servers |-> IF s = “s1” THEN FALSE ELSE TRUE]; Transition: await \A s \in (servers \ load_balancer): updated[s] = TRUE; load_balancer := servers \ load_balancer; update_flag[“s1”] := TRUE; Finish: await updated[“s1”] = TRUE; load_balancer := load_balancer \union {“s1”}; end process; … 34 Copyright © Dell Inc. All Rights Reserved.

Slide 35

Slide 35

ERRORS, LIMITATIONS, AND KNOWLEDGE Working with and around chaotic systems @nimbinatus (hachyderm) 35 Copyright © Dell Inc. All Rights Reserved.

Slide 36

Slide 36

@nimbinatus (hachyderm) Models aren’t perfect Photo by Brett Jordan on Unsplash 36 Copyright © Dell Inc. All Rights Reserved.

Slide 37

Slide 37

@nimbinatus (hachyderm) Model less complex things first Photo by Marek Okon on Unsplash 37 Copyright © Dell Inc. All Rights Reserved.

Slide 38

Slide 38

@nimbinatus (hachyderm) Models have limitations Photo by Jiawei Zhao on Unsplash 38 Copyright © Dell Inc. All Rights Reserved.

Slide 39

Slide 39

SOME FINAL THOUGHTS Putting it all together @nimbinatus (hachyderm) 39 Copyright © Dell Inc. All Rights Reserved.

Slide 40

Slide 40

@nimbinatus (hachyderm) Embrace failure 40 Copyright © Dell Inc. All Rights Reserved.

Slide 41

Slide 41

@nimbinatus (hachyderm) Trust your experience 41 Copyright © Dell Inc. All Rights Reserved.

Slide 42

Slide 42

Own work Feedback welcome! Thanks https://developer.dell.com → https://linktr.ee/nimbinatus @nimbinatus (hachyderm) 42 Copyright © Dell Inc. All Rights Reserved.