distributed_systems_MIT/lec17

lec17

dz / distributed_systems_MIT / lec17

Summary

Lecture 17: COPS, Causal Consistency

Node Tree

Nodes

COPS
content COPS
children causal_consistency, close_to_users, data_multiple_centers, larger_scheme_causality_other_channels, local_reads_writes, non_GT_COPS, only_aware_certain_relationships, situations_no_desired_order_reads_writes, spanner_similar_pattern, writes_procede_locally

data_multiple_centers
content Data in multiple centers
children data_duplicated_across
parents COPS

close_to_users
content Want: data close to users
children data_duplicated_across
parents COPS

data_duplicated_across
content Data duplicated across multiple data centers
parents data_multiple_centers, close_to_users

spanner_similar_pattern
content Spanner follows this similar pattern
parents COPS

writes_procede_locally
content Can we have a system where writes procede locally?
parents COPS

local_reads_writes
content Local reads, local writes
parents COPS

strawman_designs
content Strawman designs
children strawman_1, strawman_2

strawman_1
content Strawman 1
children good_perf, lots_of_parallelism, queue_outstanding_writes, reads_less_than_writes
parents strawman_designs, eventual_consistency

queue_outstanding_writes
content Queue of outstanding writes to send to others
parents strawman_1

good_perf
content Good performance
parents strawman_1

lots_of_parallelism
content Lots of parallelism
parents strawman_1

reads_less_than_writes
content Reads are less work than writes
parents strawman_1

eventual_consistency
content Eventual Consistency
children concurrent_writes_conflicts, strawman_1, weak_spec, which_write_is_recent, writes_stops_eventually_same (description), anomaly

writes_stops_eventually_same
content If writes stop, everything eventually has same data
parents eventual_consistency

which_write_is_recent
content Which write is most recent?
children versioning
parents eventual_consistency

versioning
content Versioning
children wall_clock
parents which_write_is_recent

wall_clock
content Wall clock
children clocks_not_in_sync, same_timestamp
parents versioning

clocks_not_in_sync
content Clocks not in sync?
children lamport_clocks (solution)
parents wall_clock

lamport_clocks
content Lamport Clocks
children t_max, timestamps_related_realtime_cope_fast_clocks (description)
parents clocks_not_in_sync

same_timestamp
content Same timestamp?
children use_id_from_datacenter
parents wall_clock

use_id_from_datacenter
content Use ID from datacenter
children placed_bottom_bits
parents same_timestamp

placed_bottom_bits
content placed in bottom bits
parents use_id_from_datacenter

anomaly
content Anomaly
parents eventual_consistency

weak_spec
content Weak spec, lots of freedom in implementation
parents eventual_consistency

timestamps_related_realtime_cope_fast_clocks
content A way to assign timestamps that are related to relatime, but cope with fast clocks.
parents lamport_clocks

t_max
content T_max: highest version it has seen
children calculate_t
parents lamport_clocks

calculate_t
content T = (T_max + 1, real time)
parents t_max

concurrent_writes_conflicts
content Concurrent writes causing conflicts
children last_writer_wins, more_sophisticated_conflict_resolution, no_good_answer
parents eventual_consistency

no_good_answer
content "real puzzle". No good answer
parents concurrent_writes_conflicts

last_writer_wins
content Last writer wins policy
parents concurrent_writes_conflicts

more_sophisticated_conflict_resolution
content More sophisticated conflict resolution
parents concurrent_writes_conflicts

strawman_2
content Strawman 2
children eventual_consistency_barriers, sync_operator
parents strawman_designs

eventual_consistency_barriers
content Eventual consistency plus barriers
children requires_careful_thought
parents strawman_2

sync_operator
content Sync Operator
children encode_order_instead_of_waiting, quorum_avoid_faults, waits_until_version
parents strawman_2

waits_until_version
content Waits until all data centers know at least up to version number
children key_version
parents sync_operator

key_version
content Key, version number
parents waits_until_version

requires_careful_thought
content Requires careful thought from the programmer to use
parents eventual_consistency_barriers

quorum_avoid_faults
content Use of quorum to avoid fault tolerance problems
parents sync_operator

encode_order_instead_of_waiting
content Instead of waiting, encode the order
children designated_log_server
parents sync_operator

designated_log_server
content Each data center has designated log server in charge of communicating to other data centers
children non_scalable_solution, sent_log_order, all_writes_through_log_server
parents encode_order_instead_of_waiting

non_scalable_solution
content non-scalable solution
children all_writes_through_log_server
parents designated_log_server

sent_log_order
content Sent in log ordre
parents designated_log_server

all_writes_through_log_server
content All writes go through log server
children eliminate_central_log
parents designated_log_server, non_scalable_solution

eliminate_central_log
content How to elimante central log server?
parents all_writes_through_log_server

non_GT_COPS
content non-GT COPS
children context_accumulates_puts_gets, get_transaction (abbrevation), cascading_wait_problem
parents COPS

get_transaction
content GT = get transaction
parents non_GT_COPS

cascading_wait_problem
content Cascading wait problem
parents non_GT_COPS

context_accumulates_puts_gets
content Context that accumulates puts/gets
children deps_in_put
parents non_GT_COPS

deps_in_put
content Dependencies in "put"
children passed_to_local_shard
parents context_accumulates_puts_gets

passed_to_local_shard
content Passed to local shard server
parents deps_in_put

causal_consistency
content Causal Consistency
children causal_dependency, dep_relationship_transitive, no_obligations_if_not_causal, actions_induce_dependencies (description)
parents COPS

actions_induce_dependencies
content Clients actions induce dependencies
children client_put_dep_prev_put, get_corr_put
parents causal_consistency

client_put_dep_prev_put
content Client put/get, get/put, or put/put depends on previous put
parents actions_induce_dependencies

get_corr_put
content Client get corresponds to a put
parents actions_induce_dependencies

dep_relationship_transitive
content Dependency relationship is transitive
parents causal_consistency

causal_dependency
content Causal Dependency
children client_reads_b_sees_a
parents causal_consistency

no_obligations_if_not_causal
content If updates not causally related, no obligations about maintaining order.
parents causal_consistency

client_reads_b_sees_a
content If A depends on B, and client reads B, the client is guaranteed to see A
parents causal_dependency

only_aware_certain_relationships
content Only aware of certain relationships
parents COPS

larger_scheme_causality_other_channels
content In the larger scheme: causality is often conveyed through channels not aware of by COPS
parents COPS
remarks this was kind of vague in the lecture. There was an example given about a human operator calling another human to explicitely look at a node because it updated.

situations_no_desired_order_reads_writes
content Situations where no one order of reads or writes will yield desired behavior.
children ACL, transactions (mechanism used to help with this (I think?)), at_least_new_as_deps
parents COPS

transactions
content Transactions
parents situations_no_desired_order_reads_writes

ACL
content ACL (Access Control List)
children mutually_consistent_ACL
parents situations_no_desired_order_reads_writes

at_least_new_as_deps
content At least as new as dependencies
parents situations_no_desired_order_reads_writes

mutually_consistent_ACL
content How to get mutually consistent ACL?
children COPS_GT (solution)
parents ACL

COPS_GT
content COPS GT
parents mutually_consistent_ACL
remarks Not elaborated on in lecture (out of time)