*a side-by-side reference sheet*

grammar and invocation | variables and expressions | arithmetic and logic | strings | regexes | dates and time | tuples | arrays | arithmetic sequences | multidimensional arrays | dictionaries | functions | execution control | file handles | directories | processes and environment | libraries and namespaces | reflection

tables | import and export | relational algebra | aggregation

vectors | matrices | descriptive statistics | distributions | linear models | statistical tests | time series | curve fitting | univariate charts | bivariate charts | multivariate charts

matlab | r | numpy | |
---|---|---|---|

version used | MATLAB 8.3Octave 3.8 | 3.1 | Python 2.7NumPy 1.7SciPy 0.13Pandas 0.12Matplotlib 1.3 |

show version | $ matlab -nojvm -nodisplay -r 'exit' $ octave --version | $ r --version | sys.version np.__version__ sp.__version__ mpl.__version__ |

implicit prologue | none | install.packages('ggplot2') library('ggplot2') | import sys, os, re, math import numpy as np import scipy as sp import scipy.stats as stats import pandas as pd import matplotlib as mpl import matplotlib.pyplot as plt |

grammar and invocation | |||

matlab | r | numpy | |

interpreter | $ cat >>foo.m 1 + 1 exit $ matlab -nojvm -nodisplay -r "run('foo.m')" $ octave foo.m | $ cat >>foo.r 1 + 1 $ Rscript foo.r $ r -f foo.r | $ cat >>foo.py print(1 + 1) $ python foo.py |

repl | $ matlab -nojvm -nodisplay $ octave | $ r | $ python |

command line program | $ matlab -nojvm -nodisplay -r 'disp(1 + 1); exit' $ octave --silent --eval '1 + 1' | $ Rscript -e 'print("hi")' | python -c 'print("hi")' |

block delimiters | function end if elseif else end while end for end | { } | offside rule |

statement separator | ; or newlineNewlines not separators after three dots: ...Output is suppressed when lines end with a semicolon. | ; or sometimes newlineNewlines not separators inside (), [], {}, '', "", or after binary operator. | newline or ;Newlines not separators inside (), [], {}, triple quote literals, or after backslash: \ |

end-of-line comment | 1 + 1 % addition | 1 + 1 # addition | 1 + 1 # addition |

variables and expressions | |||

matlab | r | numpy | |

assignment | i = 3 | i = 3 i <- 3 3 -> i assign("i", 3) | i = 3 |

compound assignmentarithmetic, string, logical | none | none | # do not return values: += -= *= /= //= %= **= += *= &= |= ^= |

null | % only used in place of numeric values: NaN | NA NULL | None |

null test | isnan(v) % true for '', []: isempty(v) | is.na(v) is.null(v) | v == None v is None |

conditional expression | none | (if (x > 0) x else -x) ifelse(x > 0, x, -x) | x if x > 0 else -x |

arithmetic and logic | |||

matlab | r | numpy | |

true and false | 1 0 true false | TRUE FALSE T F | True False |

falsehoods | false 0 0.0matrices evaluate to false unless nonempty and all entries evaluate to true | FALSE F 0 0.0matrices evaluate to value of first entry; string in boolean context causes error | False None 0 0.0 '' [] {} |

logical operators | ~true | (true & false) % short-circuit operators: && || | !TRUE | (TRUE & FALSE)short-circuit operators:&& || & and | can operate on and return vectors, but && and || return scalars | and or not |

relational operators | == ~= > < >= <= | == != > < >= <= | == != > < >= <= |

arithmetic operatorsadd, sub, mult, div, quot, rem | + - * / none mod(n, divisor) | + - * / %/% %% | + - * / // % |

integer division | fix(13 / 5) | 13 %/% 5 as.integer(13 / 5) | 13 // 5 |

integer division by zero | Inf NaN or -Inf | result of converting Inf or NaN to an integer with as.integer:NA | raises ZeroDivisionError |

float division | 13 / 5 | 13 / 5 | float(13) / 5 |

float division by zerodividend is positive, zero, negative | these values are literals:Inf NaN -Inf | these values are literals:Inf NaN -Inf | raises ZeroDivisionError |

power | 2 ^ 16 | 2 ^ 16 2 ** 16 | 2 ** 16 |

sqrt | sqrt(2) | sqrt(2) | math.sqrt(2) |

sqrt(-1) | % returns 0 + 1i: sqrt(-1) | # returns NaN: sqrt(-1) # returns 0+1i: sqrt(-1+0i) | # raises ValueError: math.sqrt(-2) # returns 1.41421j: import cmath cmath.sqrt(-2) |

transcendental functions | exp log sin cos tan asin acos atan atan2 | exp log sin cos tan asin acos atan atan2 | math.exp math.log math.sin math.cos math.tan math.asin math.acos math.atan math.atan2 |

transcendental constants | pi e | pi exp(1) | math.pi math.e |

float truncationround towards zero, to nearest integer, down, up | fix(x) round(x) floor(x) ceil(x) | as.integer(x) round(x) floor(x) ceiling(x) | int(x) int(round(x)) math.floor(x) math.ceil(x) |

absolute valueand signum | abs sign | abs sign | abs(-3.7) math.copysign(1, -3.7) |

integer overflow | becomes float; largest representable integer in the variable intmax | becomes float; largest representable integer in the variable .Machine$integer.max | becomes arbitrary length integer of type long |

float overflow | Inf | Inf | raises OverflowError |

float limits | eps realmax realmin | .Machine$double.eps .Machine$double.xmax .Machine$double.xmin | np.finfo(np.float64).eps np.finfo(np.float64).max np.finfo(np.float64).min |

complex construction | 1 + 3i | 1 + 3i | 1 + 3j |

complex decomposition | real imag abs arg conj | Re Im abs Arg Conj | import cmath z.real z.imag cmath.polar(z)[1] |

random numberuniform integer, uniform float | floor(100 * rand) rand | floor(100 * runif(1)) runif(1) | np.random.randint(0, 100) np.random.rand() |

random seedset, get, and restore | rand('state', 17) sd = rand('state') rand('state', sd) | set.seed(17) sd = .Random.seed none | np.random.seed(17) sd = np.random.get_state() np.random.set_state(sd) |

bit operators | bitshift(100, 3) bitshift(100, -3) bitand(1, 2) bitor(1, 2) bitxor(1, 2) % MATLAB: bitcmp(1, 'uint16') % Octave: bitcmp(1, 16) | none | 100 << 3 100 >> 3 1 & 2 1 | 2 1 ^ 2 ~1 |

strings | |||

matlab | r | numpy | |

literal | 'don''t say "no"' % Octave only: "don't say \"no\"" | "don't say \"no\"" 'don\'t say "no"' | 'don\'t say "no"' "don't say \"no\"" r"don't " r'say "no"' |

newline in literal | no | yes | no |

literal escapes | % Octave double quote only: \\ \" \' \0 \a \b \f \n \r \t \v | \\ \" \' \a \b \f \n \r \t \v \ooo | # single and double quoted: \ newline \\ \' \" \a \b \f \n \r \t \v \ooo \xhh |

concatenate | strcat('one ', 'two ', 'three') | paste("one ", "two ", "three") | 'one ' + 'two ' + 'three'literals, but not variables, can be concatenated with juxtaposition:'one ' "two " 'three' |

replicate | hbar = repmat('-', 1, 80) | hbar = paste(rep('-', 80), collapse='') | hbar = '-' * 80 |

index of substring | % returns array of one-indexed % locations strfind('hello', 'el') | counts from one, returns-1 if not found regexpr("el", "hello") | counts from zero, raises ValueError if not found:'hello'.index('el') |

extract substring | s = 'hello' % syntax error: 'hello'(1:4) s(1:4) | substr("hello", 1, 4) | 'hello'[0:4] |

split | % returns cell array: strsplit('foo,bar,baz', ',') | strsplit('foo,bar,baz', ',') | 'foo,bar,baz'.split(',') |

join | % takes cell array as arg: strjoin({'foo', 'bar', 'baz'}, ',') | paste("foo", "bar", "baz", sep=",") paste(c('foo', 'bar', 'baz'), collapse=',') | ','.join(['foo', 'bar', 'baz']) |

trimboth sides, left, right | strtrim(' foo ')nonedeblank('foo ') | gsub("(^[\n\t ]+|[\n\t ]+$)", "", " foo ") sub("^[\n\t ]+", "", " foo") sub("[\n\t ]+$", "", "foo ") | ' foo '.strip() ' foo'.lstrip() 'foo '.rstrip() |

padon right, on left, centered | s = repmat(' ', 1, 10) s(1:5) = 'lorem' strjust(s, 'left') strjust(s, 'right') strjust(s, 'center') | sprintf("%-10s", "lorem") sprintf("%10s", "lorem") none | 'lorem'.ljust(10) 'lorem'.rjust(10) 'lorem'.center(10) |

number to string | strcat('value: ', num2str(8)) | paste("value: ", toString("8")) | 'value: ' + str(8) |

string to number | 7 + str2num('12') 73.9 + str2num('.037') | 7 + as.integer("12") 73.9 + as.double(".037") | 7 + int('12') 73.9 + float('.037') |

translate case | lower('FOO') upper('foo') | tolower("FOO") toupper("foo") | 'foo'.upper() 'FOO'.lower() |

sprintf | sprintf('%s: %.3f %d', 'foo', 2.2, 7) | sprintf("%s: %.3f %d", "foo", 2.2, 7) | '%s: %.3f %d' % ('foo', 2.2, 7) |

length | length('hello') | nchar("hello") | len('hello') |

character access | s = 'hello' % syntax error: 'hello'(1) s(1) | substr("hello", 1, 1) | 'hello'[0] |

chr and ord | char(65) double('A') | intToUtf8(65) utf8ToInt("A") | chr(65) ord('A') |

regular expressions | |||

matlab | r | numpy | |

character class abbreviations | . \d \D \s \S \w \W % also C-string style backslash escapes: \a \b \f \n \r \t \v | # escape backslash in strings by doubling: . \d \D \s \S \w \W | . \d \D \s \S \w \W |

anchors | ^ $ \< \> | # escape backslash in strings by doubling: ^ $ \< \> \b \B | ^ $ \A \b \B \Z |

match test | regexp('hello', '^[a-z]+$') regexp('hello', '^\S+$') | regexpr("^[a-z]+$", "hello") > 0 regexpr('^\\S+$', "hello") > 0 | re.search(r'^[a-z]+$', 'hello') re.search(r'^\S+$', 'hello') |

case insensitive match test | regexpi('Lorem Ipsum', 'lorem') | regexpr('(?i)lorem', "Lorem Ipsum") > 0 | re.search(r'lorem', 'Lorem Ipsum', re.I) |

substitutionfirst match, all matches | regexprep('do re mi mi mi', 'mi', 'ma', 'once') regexprep('do re mi mi mi', 'mi', 'ma') | sub('mi', 'ma', 'do re mi mi mi') gsub('mi', 'ma', 'do re mi mi mi') | rx = re.compile(r'mi') s = rx.sub('ma', 'do re mi mi mi', 1) s2 = rx.sub('ma', 'do re mi mi mi') |

backreference in match and substitution | regexp('do do', '(\w+) \1') regexprep('do re', '(\w+) (\w+)', '$2 $1') | regexpr('(\\w+) \\1', 'do do') > 0 sub('(\\w+) (\\w+)', '\\2 \\1', 'do re') | nonerx = re.compile(r'(\w+) (\w+)') rx.sub(r'\2 \1', 'do re') |

dates and time | |||

matlab | r | numpy | |

current date/time | t = now | t = as.POSIXlt(Sys.time()) | import datetime t = datetime.datetime.now() |

date/time type | floating point number representing days since year 0 in the Gregorian calendar | POSIXlt | datettime |

date/time difference type | floating point number representing days | a difftime object which behaves like a floating point number representing seconds | timedelta, which can be converted to float value in seconds via total_seconds() method |

get date parts | dv = datevec(t) dv(1) dv(2) dv(3) % syntax error: datevec(t)(1) | t$year + 1900 t$mon + 1 t$mday | t.year t.month t.day |

get time parts | dv = datevec(t) dv(4) dv(5) dv(6) | t$hour t$min t$sec | t.hour t.minute t.second |

build date/time from parts | t = datenum([2011 9 20 23 1 2]) | t = as.POSIXlt(Sys.time()) t$year = 2011 - 1900 t$mon = 9 - 1 t$mday = 20 t$hour = 23 t$min = 1 t$sec = 2 | import datetime t = datetime.datetime(2011, 9, 20, 23, 1, 2) |

convert to string | datestr(t) | print(t) | str(t) |

strptime | s = '2011-09-20 23:01:02' fmt = 'yyyy-mm-dd HH:MM:SS' t = datenum(s, fmt) | t = strptime('2011-09-20 23:01:02', '%Y-%m-%d %H:%M:%S') | import datetime s = '2011-05-03 10:00:00' fmt = '%Y-%m-%d %H:%M:%S' t = datetime.datetime.strptime(s, fmt) |

strftime | datestr(t, 'yyyy-mm-dd HH:MM:SS') | format(t, format='%Y-%m-%d %H:%M:%S') | t.strftime('%Y-%m-%d %H:%M:%S') |

tuples | |||

matlab | r | numpy | |

type | cell | list | tuple |

literal | tup = {1.7, 'hello', [1 2 3]} | tup = list(1.7, "hello", c(1, 2, 3)) | tup = (1.7, "hello", [1,2,3]) |

lookup element | % indices start at one: tup{1} | # indices start at one: tup[[1]] | # indices start at zero: tup[0] |

update element | tup{1} = 2.7 | tup[[1]] = 2.7 | tuples are immutable |

length | length(tup) | length(tup) | len(tup) |

arrays | |||

matlab | r | numpy | |

element type | always numeric | # "numeric": class(c(1, 2, 3)) # arrays can also have "boolean" or "string" elements | # values can have different types: [type(x) for x in a] |

literal | a = [1, 2, 3, 4] % commas are optional: a = [1 2 3 4] | # use c() constructor: a = c(1, 2, 3, 4) | a = [1, 2, 3, 4] |

size | length(a) | length(a) | len(a) |

empty test | length(a) == 0 % An array used in a conditional test is % false unless nonempty and all entries evaluate % as true. | length(a) == 0 | not a |

lookup | % indices start at one: a(1) | # indices start at one a[1] | # indices start at zero a[0] |

update | a(1) = -1 | a[1] = -1 | a[0] = -1 |

out-of-bounds behavior | a = [] % error: a(1) % increases array size to 10; % zero-fills slots 1 through 9: a(10) = 10 | a = c() # evaluates as NA: a[10] # increases array size to 10: a[10] = "lorem" | a = [] # raises IndexError: a[10] # raises IndexError: a[10] = 'lorem' |

index of element | a = [7 8 9 10 8] % returns [2 5]: find(a == 8) % returns 2: find(a == 8, 1, 'first') | a = c('x', 'y', 'z', 'w', 'y') # c(2, 5): which(a == 'y') | a = ['x', 'y', 'z', 'w', 'y'] a.index('y') # 1 a.rindex('y') # 4 |

sliceby endpoints | a = ['a' 'b' 'c' 'd' 'e'] % returns ['c' 'd']: a(3:4) | a = c("a", "b", "c", "d", "e") # return c("c", "d"): a[seq(3, 4)] | a = ['a', 'b', 'c', 'd', 'e'] # return ['c', 'd']: a[2:4] |

slice to end | a(2:end) | tail(a, n=length(a) - 1) | a[1:] |

integer array as index | [1 2 3]([1 3 3]) | c(1, 2, 3)[c(1, 3, 3)] | np.array([1, 2,3])[[0, 2, 2]] |

logical array as index | [1 2 3]([true false true]) | c(1, 2, 3)[c(T, F, T)] | np.array([1, 2, 3])[[True, False, True]] |

concatenate | a = [1 2 3] a2 = [a [4 5 6]] a = [a [4 5 6]] % or: a = horzcat(a, a2) | a = c(1, 2, 3) a2 = append(a, c(4, 5, 6)) a = append(a, c(4, 5, 6)) | a = [1, 2, 3] a2 = a + [4, 5, 6] a.extend([4, 5, 6]) |

replicate | a = repmat(NA, 1, 10) | a = rep(NA, 10) rep(c("a", "b", "c"), c(30, 50, 90)) | a = [None] * 10 a = [None for i in range(0, 10)] |

copyaddress copy, shallow copy, deep copy | There is no address copy. Because arrays cannot be nested, there is no distinction between shallow copy and deep copy. Assignment and passing an array to a function can be regarded as performing a shallow or deep copy, though MATLAB does not allocate memory for a 2nd array until one of the arrays is modified. | Arrays in R behave like arrays in MATLAB. | import copy a = [1, 2, [3, 4]] a2 = a a3 = list(a) a4 = copy.deepcopy(a) |

iteration | a = [9 7 3] for i = 1:numel(a) x = a(i) disp(x) end | for (x in c(9, 7, 3)) { print(x) } | for i in [9, 7, 3]: print(i) |

indexed iteration | for (i in seq_along(a)) { cat(sprintf("%s at index %d\n", i, a[i])) } | a = ['do', 're', 'mi', 'fa'] for i, s in enumerate(a): print('%s at index %d' % (s, i)) | |

reverse | a = [1 2 3] a2 = fliplr(a) a = fliplr(a) | a = c(1, 2, 3) a2 = rev(a) a = rev(a) | a = [1, 2, 3] a2 = a[::-1] a.reverse() |

sort | a = [3 1 4 2] a = sort(a) | a = c('b', 'A', 'a', 'B') a2 = sort(a) a = sort(a) | a = ['b', 'A', 'a', 'B'] sorted(a) a.sort() a.sort(key=str.lower) |

dedupe | a = [1 2 2 3] a2 = unique(a) | a = c(1, 2, 2, 3) a2 = unique(a) | a = [1, 2, 2, 3] a2 = list(set(a))) |

membership | ismember(7, a) | 7 %in% a is.element(7, a) | 7 in a |

intersection | intersect([1 2], [2 3 4]) | intersect(c(1, 2), c(2, 3, 4)) | {1, 2} & {2, 3, 4} |

union | union([1 2], [2 3 4]) | union(c(1, 2), c(2, 3, 4)) | {1, 2} | {2, 3, 4} |

relative complement, symmetric difference | setdiff([1 2 3], [2]) a1 = [1 2] a2 = [2 3 4] union(setdiff(a1, a2), setdiff(a2, a1)) | setdiff(c(1, 2, 3), c(2)) union(setdiff(c(1, 2), c(2, 3, 4)), setdiff(c(2, 3, 4), c(1, 2))) | {1, 2, 3} - {2} {1, 2} ^ {2, 3, 4} |

map | arrayfun( @(x) x*x, [1 2 3]) | sapply(c(1,2,3), function (x) { x * x}) | map(lambda x: x * x, [1, 2, 3]) # or use list comprehension: [x * x for x in [1, 2, 3]] |

filter | a = [1 2 3] a(a > 2) | a = c(1, 2, 3) a[a > 2] Filter(function(x) { x > 2}, a) | filter(lambda x: x > 1, [1, 2, 3]) # or use list comprehension: [x for x in [1, 2, 3] if x > 1] |

reduce | Reduce(function(x, y) { x + y }, c(1, 2, 3), 0) | reduce(lambda x, y: x + y, [1 ,2, 3], 0) | |

universal and existential tests | all(mod([1 2 3 4], 2) == 0) any(mod([1 2 3 4]) == 0) | all(c(1, 2, 3, 4) %% 2 == 0) any(c(1, 2, 3, 4) %% 2 == 0) | all(i % 2 == 0 for i in [1, 2, 3, 4]) any(i % 2 == 0 for i in [1, 2, 3, 4]) |

shuffle and sample | a = c(1, 1, 2, 3, 9, 28) sample(a, 3) a[sample.int(length(a))] | from random import shuffle, sample a = [1, 2, 3, 4] shuffle(a) sample(a, 2) | |

zip | none; MATLAB arrays can't be nested | # R arrays can't be nested. # One approximation of zip is a 2d array: a = rbind(c(1, 2, 3), c('a', 'b', 'c')) # To prevent data type coercion, use a data frame: df = data.frame(numbers=c(1, 2, 3), letters=c('a', 'b', 'c')) | # array of 3 pairs: a = zip([1, 2, 3], ['a', 'b', 'c']) |

arithmetic sequences | |||

matlab | r | numpy | |

unit difference | 1:100 | # type integer: 1:100 seq(1, 100) # type double: seq(1, 100, 1) | range(1, 101) |

difference of 10 | 0:10:100 | # type double: seq(0, 100, 10) | range(0, 101, 10) |

difference of 0.1 | 0:0.1:10 | seq(0, 10, 0.1) | [0.1 * x for x in range(0, 101)]3rd arg is length of sequence, not step size:sp.linspace(0, 10, 100) |

iterate over arithmetic sequence | range replaces xrange in Python 3:for i in xrange(1, 1000001): code | ||

convert arithmetic sequence to array | a = range(1, 11)Python 3:a = list(range(1, 11)) | ||

multidimensional arrays | |||

matlab | r | numpy | |

element type | always numeric | a = array(c(1, 2, 3)) # "array": class(a) # "boolean", "numeric", or "string": class(c(a)) | np.array([1, 2, 3]).dtype # possible values: np.bool, np.int64, np.float64, # np.complex128, ... |

literal—1d | [1, 2, 3] % commas are optional: [1 2 3] | # no literal; pass array to multidimensional # array constructor: array(c(1, 2, 3)) | # no literal; pass sequence type to multidimsional # array constructor: np.array([1, 2, 3]) np.array((1, 2, 3)) |

literal—2d | [1, 2; 3, 4] % commas optional; newlines can replace semicolons:: [1 2 3 4] | none | none |

construct from sequence—2d | reshape([1 2 3 4], 2, 2) | array(c(1, 2, 3, 4), dim=c(2, 2)) | np.array([1, 2, 3, 4]).reshape(2, 2) |

construct from sequence—3d | reshape([1 2 3 4 5 6 7 8], 2, 2, 2) | array(c(1, 2, 3, 4, 5, 6, 7, 8), dim=c(2, 2, 2)) | np.array([1, 2, 3, 4, 5, 6, 7, 8]).reshape(2, 2, 2) |

construct from nested sequences—2d | none | none | np.array([[1, 2], [3, 4]]) |

construct from nested sequences—3d | none | none | np.array([[[1, 2], [3, 4]], [[5, 6], [7, 8]]]) |

construct from rows—2d | row1 = [1 2 3] row2 = [4 5 6] a = [row1; row2] | rbind(c(1, 2, 3), c(4, 5, 6)) | np.vstack((np.array([1, 2, 3]), np.array([4, 5, 6]))) np.array([[1, 2], [3, 4]]) |

construct from columns—2d | col1 = [1; 4] col2 = [2; 5] col3 = [3; 6] % commas are optional: a = [col1, col2, col3] | cbind(c(1, 4), c(2, 5), c(3, 6)) | cols = ( np.array([1, 4]), np.array([2, 5]), np.array([3, 6]) ) np.vstack(cols).transpose() |

construct from subarrays—2d | m = [1 3; 2 4] m4_by_2 = [m; m] m2_by_4 = [m m] | m = matrix(c(1, 2, 3, 4), nrow=2) m4_by_2 = rbind(m, m) m2_by_4 = cbind(m, m) | |

construct 3d array from 2d arrays | a = [1, 2; 3, 4] a(:,:,2) = [5, 6; 7, 8] | ||

sizenumber of elements, number of dimensions, dimension lengths | numel(a) ndims(a) size(a) % length of 1st dimension (i.e. # of rows): size(a, 1) % length of longest dimension: length(a) | length(a) length(dim(a)) dim(a) | a.size a.ndim a.shape # number of rows: len(a) |

lookup—1d | indices start at one:[1 2 3](1) | indices start at one:array(c(1, 2, 3))[1] | indices start at zero:a = np.array([1, 2, 3]) a[0] |

lookup—2d | [1 2; 3 4](1, 1) | a = array(c(1, 2, 3, 4), dim=c(2, 2) a[1, 1] | a = np.array([[1, 2], [3, 4]]) a[0][0] ora[0, 0] |

1d lookup on 2d array | a = [2 4; 6 8] % returns 8: a(4) % convert to column vector of length 4: a2 = a(:) | a = array(c(2, 4, 6, 8), dim=c(2, 2)) # returns 8: a[4] | a = np.array([[2, 4], [6, 8]]) # returns np.array([6, 8]): a[1] # returns 8: a.reshape(4)[3] |

update—2d | a = [2 4; 6 8] a(1, 1) = 3 | ||

out-of-bounds behavior | a = [2 4; 6 8] % error: x = a(3, 1) % becomes 3x2 array with zero at (3, 2): a(3, 1) = 9 | ||

slicerow, column | a = [1 2 3; 4 5 6; 7 8 9] % 2nd row: a(2, :) % 2nd column: a(:, 2) | ||

slice to end | a = [1 2 3 4] % [2 3 4]: a(2:end) | ||

slice subarray | a = [1 2 3; 4 5 6; 7 8 9] % top left 2x2 subarray: a(1:2, 1:2) % bottom right 2x2 subarray: a(end-1:end, end-1:end) % 2x2 array containing corners: a([1 3], [1 3]) | ||

transpose | a = [1 2; 3 4] a' transpose(a) | a = array(c(1, 2, 3, 4), dim=c(2, 2)) t(a) | a = np.array([[1, 2], [3, 4]]) a.transpose() a.T |

swap axes | a = reshape([1 2 3 4 5 6 7 8], 2, 2, 2) % swap 2nd and 3rd axes: permute(a, [1 3 2]) | ||

flip—2d | % [ 2 1; 4 3]: fliplr([1 2; 3 4]) % [3 4; 1 2]: flipud([1 2; 3 4]) | ||

flip—3d | a = reshape([1 2 3 4 5 6 7 8], 2, 2, 2) flipdim(a, 3) | ||

circular shift—2d | circshift | ||

rotate—2dclockwise, counter-clockwise | a = [1 2; 3 4] % [3 1; 4 2]: rot90(a, -1) % [2 4; 1 3]: rot90(a) | ||

apply function element-wise | arrayfun | np.vectorize | |

apply function to linear subarraysrows, columns | arrayfunnote return value type | ||

dictionaries | |||

matlab | r | numpy | |

literal | % no literal; use constructor: d = struct('n', 10, 'avg', 3.7, 'sd', 0.4) % or build from two cell arrays: d = cell2struct({10 3.7 0.4}, {'n' 'avg' 'sd'}, 2) | # keys are 'n', 'avg', and 'sd': d = list(n=10, avg=3.7, sd=0.4) # keys are 1, 2, and 3: d2 = list('do', 're', 'mi') | d = {'n': 10, 'avg': 3.7, 'sd': 0.4} |

size | length(fieldnames(d)) | length(d) | len(d) |

lookup | d.n getfield(d, 'n') | d[['n']] # if 'n' is a key: d$n | d['n'] |

update | d.var = d.sd**2 | d$var = d$sd**2 | d['var'] = d['sd']**2 |

missing key behavior | error | NULL | raises KeyError |

is key present | isfield(d, 'var') | is.null(d$var) | 'var' in d |

delete | d = rmfield(d, 'sd') | d$sd = NULL | del(d['sd']) |

iterate | for i = 1:numel(fieldnames(d)) k = fieldnames(d){i} v = d.(k) codeend | for (k in names(d)) { v = d[[k]] code} | for k, v in d.iteritems():code |

keys and values as arrays | % these return cell arrays: fieldnames(d) struct2cell(d) | names(d) unlist(d, use.names=F) | d.keys() d.values() |

merge | none | d1 = list(a=1, b=2) d2 = list(b=3, c=4) # values of first dictionary take precedence: d3 = c(d1, d2) | d1 = {'a':1, 'b':2} d2 = {'b':3, 'c':4} d1.update(d2) |

functions | |||

matlab | r | numpy | |

define function | function add(x, y) x + y end | add = function(x, y) {x + y} | |

invoke function | add(3, 7) | add(3, 7) | |

nested function | function ret1 = foo(x, y) function ret2 = bar(x, y) ret2 = x + y; end ret1 = bar(x, y); end | foo = function(x, y) { bar = function(x, y) { x + y } bar(x, y) } | |

missing argument behavior | raises error if code with the parameter that is missing an argument is executed | raises error | |

extra argument behavior | ignored | raises error | |

default argument | function mylog(x, base=10) log(x) / log(base) end | mylog = function(x,base=10) { log(x) / log(base) } | |

variadic function | function s = add(varargin) if nargin == 0 s = 0 else r = add(varargin{2:nargin}) s = varargin{1} + r end end | add = function (...) { a = list(...) if (length(a) == 0) return(0) s = 0 for(i in 1:length(a)) { s = s + a[[i]] } return(s) } | |

return value | function ret = add(x, y) ret = x + y; end % If a return variable is declared, the value assigned % to it is returned. Otherwise the value of the last % statement will be used if it does not end with a % semicolon. | return argument or last expression evaluated. NULL if return called without an argument. | |

multiple return values | function [x, y] = first_two(a) x = a(1); y = a(2); end % sets first to 7; second to 8: [first, second] = first_two([7 8 9]) | ||

anonymous function literal | % body must be an expression: @(a,b) a+b | function(a,b) {a+b} | |

invoke anonymous function | |||

closure | make_counter = function() { i = 0 function() { i <<- i + 1 i } } | ||

function as value | @add | add | |

overload operator | |||

invoke operator like function | `+`(3, 7) | ||

execution control | |||

matlab | r | numpy | |

if | if (x > 0) disp('positive') elseif (x < 0) disp('negative') else disp('zero') end | if (x > 0) { print('positive') } else if (x < 0) { print('negative') } else { print('zero') } | if x > 0: print('positive') elif x < 0: print('negative') else: print('zero') |

while | i = 0 while (i < 10) i = i + 1 disp(i) end | while (i < 10) { i = i + 1 print(i) } | while i < 10: i += 1 print(i) |

for | for i = 1:10 disp(i) end | for (i in 1:10) { print(i) } | for i in range(1,11): print(i) |

break/continue | break continue | break next | break continue |

raise exception | error('%s', 'failed') | stop('failed') | raise Exception('failed') |

handle exception | try error('failed') catch err disp(err) end | tryCatch( stop('failed'), error=function(e) print(message(e))) | try: raise Exception('failed') except Exception as e: print(e) |

file handles | |||

matlab | r | numpy | |

standard file handles | stdin stdout stderr | stdin() stdout() stderr() | sys.stdin sys.stdout sys.stderr |

read line from stdin | line = input('', 's') | line = readLines(n=1) | line = sys.stdin.readline() |

write line to stdout | puts('hello\n') | cat("hello\n") writeLines("hello") | print('hello') |

write formatted string to stdout | printf('%.2f\n', pi) | cat(sprintf("%.2f\n", pi)) | import math print('%.2f' % math.pi) |

open file for reading | if ((f = fopen('/etc/hosts')) == -1) error('failed to open file') endif | f = file("/etc/hosts", "r") | f = open('/etc/hosts') |

open file for writing | if ((f = fopen('/tmp/test', 'w') == -1) error('failed to open file') endif | f = file("/tmp/test", "w") | f = open('/tmp/test', 'w') |

open file for appending | if ((f = fopen('/tmp/err.log', 'a') == -1) error('failed to open file') endif | f = file("/tmp/err.log", "a") | f = open('/tmp/err.log', 'a') |

close file | fclose(f) | close(f) | f.close() |

i/o errors | fopen returns -1; fclose throws an error | raise IOError exception | |

read line | line = fgets(f) | line = readLines(f, n=1) | line = f.readline() |

iterate over file by line | while(!feof(f)) line = fgets(f) puts(line) endwhile | for line in f: print(line) | |

read file into array of strings | lines = readLines(f) | lines = f.readlines() | |

write string | fputs(f, 'lorem ipsum') | cat("lorem ipsum", file=f) | f.write('lorem ipsum') |

write line | fputs(f, 'lorem ipsum\n') | writeLines("lorem ipsum", con=f) | f.write('lorem ipsum\n') |

flush file handle | fflush(f) | flush(f) | f.flush() |

file handle positionget, set | ftell(f) % 3rd arg can be SEEK_CUR or SEEK_END fseek(f, 0, SEEK_SET) | seek(f) # sets seek point to 12 bytes after start; # origin can also be "current" or "end" seek(f, where=0, origin="start") | f.tell() f.seek(0) |

redirect stdout to file | sink("foo.txt") | ||

write variables to file | A = [1 2; 3 4] B = [4 3; 2 1] save('data.mdata', 'A', 'B') | A = matrix(c(1, 3, 2, 4), nrow=2) B = matrix(c(4, 2, 3, 1), nrow=2) save(A, B, file='data.rdata') | A = np.matrix([[1, 2], [3, 4]]) B = np.matrix([[4, 3], [2, 1]]) # Data must be of type np.array; # file will have .npz suffix: np.savez('data', A=A, B=B) |

read variables from file | % puts A and B in scope: load('data.mdata') % puts just A in scope: load('data.mdata', 'A') | # puts A and B in scope: load('data.rdata') | data = np.load('data.npz') A = data['A'] B = data['B'] |

write all variables in scope to file | save('data.txt') | save.image('data.txt') | |

directories | |||

matlab | r | numpy | |

working directoryget, set | pwd cd('/tmp') | getwd() setwd("/tmp") | os.path.abspath('.') os.chdir('/tmp') |

build pathname | file.path("/etc", "hosts") | os.path.join('/etc', 'hosts') | |

dirname and basename | dirname("/etc/hosts") basename("/etc/hosts") | os.path.dirname('/etc/hosts') os.path.basename('/etc/hosts') | |

absolute pathname | normalizePath("..") | os.path.abspath('..') | |

iterate over directory by file | % lists /etc: ls('/etc') % lists working directory: ls() | # list.files() defaults to working directory for (path in list.files('/etc')) { print(path) } | for filename in os.listdir('/etc'): print(filename) |

glob paths | glob('/etc/*') | Sys.glob('/etc/*') | import glob glob.glob('/etc/*') |

processes and environment | |||

matlab | r | numpy | |

command line arguments | % does not include interpreter name: argv() | # first arg is name of interpreter: commandArgs() # arguments after --args only: commandArgs(TRUE) | sys.argv |

environment variableget, set | getenv('HOME') setenv('PATH', '/bin') | Sys.getenv("HOME") Sys.setenv(PATH="/bin") | os.getenv('HOME') os.environ['PATH'] = '/bin' |

exit | exit(0) | quit(save="no", status=0) | sys.exit(0) |

external command | if (shell_cmd('ls -l /tmp')) error('ls failed') endif | if (system("ls -l /tmp")) { stop("ls failed") } | if os.system('ls -l /tmp'): raise Exception('ls failed') |

libraries and namespaces | |||

matlab | r | numpy | |

load library | When a function is invoked, MATLAB searches the library path for a file with the same name and a .m suffix. Other functions defined in the file are not visible outside the file. | require("foo") % or: library("foo") | import foo |

list loaded libraries | none | search() | dir() |

library search path | path() addath('~/foo') rmpath('~/foo') | .libPaths() | sys.path |

source file | run('foo.m') | source("foo.r") | none |

install package | % Octave: how to install package % downloaded from Octave-Forge: pkg install foo-1.0.0.tar.gz | install.packages("ggplot2") | $ pip install scipy |

load package library | % Octave: pkg load foo | require("foo") # or: library("foo") | import foo |

list installed packages | % Octave: pkg list | library() installed.packages() | $ pip freeze |

reflection | |||

matlab | r | numpy | |

data type | class(x) | class(x) # often more informative: str(x) | type(x) |

attributes | % if x holds an object: x | attributes(x) | [m for m in dir(x) if not callable(getattr(o,m))] |

methods | % if x holds an object: methods(x) | none; objects are implemented by functions which dispatch based on type of first arg | [m for m in dir(x) if callable(getattr(o,m))] |

variables in scope | who() % with size and type: whos() | objects() ls() # with type and description: ls.str() | dir() |

undefine variable | clear('x') | rm(v) | del(x) |

undefine all variables | clear -a | rm(list=objects()) | |

eval | eval('1 + 1') | eval(parse(text='1 + 1')) | eval('1 + 1') |

function documentation | help tan | help(tan) ?tan | math.tan.__doc__ |

list library functions | none | ls("package:moments") | dir(stats) |

search documentation | docsearch tan | ??tan | $ pydoc -k tan |

tables | |||

r | numpy | ||

construct from column arrays | sx = {'F' 'F' 'F' 'M' 'M' 'M'} ht = [69 64 67 68 72 71] wt = [148 132 142 149 167 165] cols = {'sx', 'ht', 'wt'} people = table(sx', ht', wt', 'VariableNames', cols) | # gender, height, weight of some people # in inches and lbs: sx = c("F", "F", "F", "M", "M", "M") ht = c(69, 64, 67, 68, 72, 71) wt = c(148, 132, 142, 149, 167, 165) people = data.frame(sx, ht, wt) | sx = ['F', 'F', 'F', 'F', 'M', 'M'] ht = [69, 64, 67, 66, 72, 70] wt = [150, 132, 142, 139, 167, 165] people = pd.DataFrame({'sx': sx, 'ht': ht, 'wt': wt}) |

size | height(people) width(people) | nrow(people) ncol(people) # number of rows and cols in 2-element vector: dim(people) | |

column names as array | people.Properties.VariableNames | names(people) colnames(people) | returns Index object:people.columns |

access column as array | people.ht people.(2) | # vectors: people$ht people[,2] people[['ht']] people[[2]] # 1 column data frame: people[2] | people['ht']if name does not conflict with any DataFrame attributes:people.ht |

access row as tuple | people(1,:) | # 1 row data frame: people[1, ] # list: as.list(people[1, ]) | people.ix[0] |

access datum | % height of 1st person: people(1,2) | # height of 1st person: people[1,2] | people.get_value(0, 'ht') |

order rows by column | people[order(people$ht), ] | ||

order rows by multiple columns | people[order(sx, ht), ] | ||

order rows in descending order | people[order(-people$ht), ] | ||

limit rows | people[seq(1, 3), ] | ||

offset rows | people[seq(4, 6), ] | ||

reshape | people$couple = c(1, 2, 3, 1, 2, 3) reshape(people, idvar="couple", direction="wide", timevar="sx", v.names=c("ht", "wt")) | ||

attach columns | # put columns ht, wt, and sx # in variable name search path: attach(people) sum(ht) # alternative which doesn't put columns in # search path: with(people, sum(ht)) | none | |

detach columns | detach(people) | none | |

spreadsheet editor | can edit data, in which case return value of edit must be savedpeople = edit(people) | none | |

import and export | |||

r | numpy | ||

import tab delimited | # first row defines variable names: df = read.delim('/path/to.tab') | # first row defines column names: df = pd.read_table('/path/to.tab') | |

import csv | # first row defines variable names: df = read.csv('/path/to.csv') | # first row defines column names: df = pd.read_csv('/path/to.csv') | |

set column separator | df = read.delim('/etc/passwd', sep=':', header=FALSE, comment.char='#') | # $ grep -v '^#' /etc/passwd > /tmp/passwd df = pd.read_table('/tmp/passwd', sep=':', header=None) | |

set column separator to whitespace | df = read.delim('/path/to.txt', sep='') | df = read_table('/path/to.txt', sep='\s+') | |

set quote character | default quote character for both read.csv and read.delim is double quotes. The quote character is escaped by doubling it.# use single quote as quote character: df = read.csv('/path/to/single-quote.csv', quote="'") # no quote character: df = read.csv('/path/to/no-quote.csv', quote="") | Both read_table and read_csv use double quotes as the quote character and there is no way to change it. A double quote can be escaped by doubling it. | |

import file w/o header | # column names are V1, V2, … read.delim('/etc/passwd', sep=':', header=FALSE, comment.char='#') | # $ grep -v '^#' /etc/passwd > /tmp/passwd # # column names are X0, X1, … df = pd.read_table('/tmp/passwd', sep=':', header=None) | |

set column names | df = read.csv('/path/to/no-header.csv', header=FALSE, col.names=c('ht', 'wt', 'age')) | df = pd.read_csv('/path/to/no-header.csv', names=['ht', 'wt', 'age']) | |

set column types | # possible values: NA, 'logical', 'integer', 'numeric', # 'complex', 'character', 'raw', 'factor', 'Date', # 'POSIXct' # # If type is set to NA, actual type will be inferred to be # 'logical', 'integer', 'numeric', 'complex', or 'factor' # df = read.csv('/path/to/data.csv', colClasses=c('integer', 'numeric', 'character')) | ||

recognize null values | df = read.csv('/path/to/data.csv', colClasses=c('integer', 'logical', 'character'), na.strings=c('nil')) | df = read_csv('/path/to/data.csv', na_values=['nil']) | |

change decimal mark | df = read.csv('/path/to.csv', dec=',') | ||

recognize thousands separator | none | df = read_csv('/path/to.csv', thousands='.') | |

unequal row length behavior | Missing fields will be set to NA unless fill is set to FALSE. If the column is of type character then the fill value is an empty string ''. flush If there are extra fields they will be parsed as an extra row unless is set to FALSE | ||

skip comment lines | df = read.delim('/etc/passwd', sep=':', header=FALSE, comment.char='#') | none | |

skip rows | df = read.csv('/path/to/data.csv', skip=4) | df = read_csv('/path/to/data.csv', skiprows=4) # rows to skip can be specified individually: df = read_csv('/path/to/data.csv', skiprows=range(0, 4)) | |

max rows to read | df = read.csv('/path/to/data.csv', nrows=4) | df = read_csv('/path/to/data.csv', nrows=4) | |

index column | none | df = pd.read_csv('/path/to.csv', index_col='key_col') # hierarchical index: df = pd.read_csv('/path/to.csv', index_col=['col1', 'col2']) | |

export tab delimited | write.table(df, '/tmp/data.tab', sep='\t') | ||

export csv | # first column contains row names unless row.names # set to FALSE write.csv(df, '/path/to.csv', row.names=F) | ||

relational algebra | |||

matlab | r | numpy | |

project columns by name | people[c('sx', 'ht')] | people[['sx', 'ht']] | |

project columns by position | people[c(1, 2)] | ||

project expression | convert to cm and kg:transform(people, ht=2.54*ht, wt=wt/2.2) | ||

project all columns | people[people$ht > 66, ] | ||

rename columns | colnames(people) = c('gender', 'height', 'weight') | ||

access sub data frame | data frame of first 3 rows withht and wt columns reversedpeople[1:3,c(1,3,2)] | ||

select rows | subset(people, ht > 66) people[people$ht > 66, ] | people[people['ht'] > 66] | |

select distinct rows | unique(people$sx) | ||

split rows | class(x) is list: x = split(people, people$sx == 'F') data.frame only containing females: x$T | ||

inner join | pw = read.delim('/etc/passwd', sep=':', header=F, comment.char='#', col.names=c('name', 'passwd', 'uid', 'gid', 'gecos', 'home', 'shell')) grp = read.delim('/etc/group', sep=':', header=F, comment.char='#', col.names=c('name', 'passwd', 'gid', 'members')) merge(pw, grp, by.x='gid', by.y='gid') | # $ grep -v '^#' /etc/passwd > /tmp/passwd # $ grep -v '^#' /etc/group > /tmp/group pw = pd.read_table('/tmp/passwd', sep=':', header=None, names=['name', 'passwd', 'uid', 'gid', 'gecos', 'home', 'shell']) grp = pd.read_table('/tmp/group', sep=':', header=None, names=['name', 'passwd', 'gid', 'members']) pd.merge(pw, grp, left_on='gid', right_on='gid') | |

nulls as join values | |||

left join | merge(pw, grp, by.x='gid', by.y='gid', all.x=T) | pd.merge(pw, grp, left_on='gid', right_on='gid', how='left') | |

full join | merge(pw, grp, by.x='gid', by.y='gid', all=T) | pd.merge(pw, grp, left_on='gid', right_on='gid', how='outer') | |

antijoin | pw[!(pw$gid %in% grp$gid), ] | ||

cross join | merge(pw, grp, by=c()) | ||

aggregation | |||

matlab | r | numpy | |

group by column | grouped = people.groupby('sx') grouped.aggregate(np.max)['ht'] | ||

multiple aggregated values | grouped = people.groupby('sx') grouped.aggregate(np.max)[['ht', 'wt']] | ||

group by multiple columns | |||

aggregation functions | |||

nulls and aggregation functions | |||

vectors | |||

matlab | r | numpy | |

vector literal | same as array | same as array | same as array |

element-wise arithmetic operators | + - .* ./ | + - * / | + - * / |

result of vector length mismatch | raises error | values in shorter vector are recycled; warning if one vector is not a multiple length of the other | raises ValueError |

scalar multiplication | 3 * [1, 2, 3] [1, 2, 3] * 3 | 3 * c(1, 2, 3) c(1, 2, 3) * 3 | 3 * np.array([1, 2, 3]) np.array([1, 2, 3]) * 3 |

dot product | dot([1, 1, 1], [2, 2, 2]) | c(1, 1, 1) %*% c(2, 2, 2) | v1 = np.array([1, 1, 1]) v2 = np.array([2, 2, 2]) np.dot(v1, v2) |

cross product | cross([1, 0, 0], [0, 1, 0]) | v1 = np.array([1, 0, 0]) v2 = np.array([0, 1, 0]) np.cross(v1, v2) | |

norms | norm([1, 2, 3], 1) norm([1, 2, 3], 2) norm([1, 2, 3], Inf) | vnorm = function(x, t) { norm(matrix(x, ncol=1), t) } vnorm(c(1, 2, 3), "1") vnorm(c(1, 2, 3), "E") vnorm(c(1, 2, 3), "I") | v = np.array([1, 2, 3]) np.linalg.norm(v, 1) np.linalg.norm(v, 2) np.linalg.norm(v, np.inf) |

matrices | |||

matlab | r | numpy | |

literal or constructor | % row-major order: A = [1, 2; 3, 4] B = [4 3 2 1] | # column-major order: A = matrix(c(1, 3, 2, 4), 2, 2) B = matrix(c(4, 2, 3, 1), nrow=2) | # row-major order: A = np.matrix([[1, 2], [3, 4]]) B = np.matrix([[4, 3], [2, 1]]) |

constant matricesall zeros, all ones | zeros(3, 3) or zeros(3)ones(3, 3) or ones(3) | matrix(0, 3, 3) matrix(1, 3, 3) | |

diagonal matricesand identity | diag([1, 2, 3]) % 3x3 identity: eye(3) | diag(c(1, 2, 3) # 3x3 identity: diag(3) | |

dimensions | rows(A) columns(A) | dim(A)[1] dim(A)[2] | |

element access | A(1, 1) | A[1, 1] | A[0, 0] |

row access | A(1, 1:2) | A[1, ] | A[0] |

column access | A(1:2, 1) | A[, 1] | |

submatrix access | C = [1, 2, 3; 4, 5, 6; 7, 8, 9] C(1:2, 1:2) | C = matrix(seq(1, 9), 3, 3, byrow=T) C[1:2, 1:2] | |

scalar multiplication | 3 * A A * 3 also:3 .* A A .* 3 | 3 * A A * 3 | 3 * A A * 3 |

element-wise operators | .+ .- .* ./ | + - * / | + - np.multiply() np.divide() |

multiplication | A * B | A %*% B | A * B |

power | A ^ 3 % power of each entry: A .^ 3 | A ** 3 | |

kronecker product | kron(A, B) | kronecker(A, B) | np.kron(A, B) |

comparison | all(all(A==B)) any(any(A!=A)) | all(A==B) any(A!=B) | |

norms | norm(A, 1) norm(A, 2) norm(A, Inf) norm(A, 'fro') | norm(A, "1")??norm(A, "I") norm(A, "F") | |

transpose | transpose(A) A' | t(A) | A.transpose() |

conjugate transpose | A = [1i, 2i; 3i, 4i] A' | A = matrix(c(1i, 2i, 3i, 4i), nrow=2, byrow=T) Conj(t(A)) | A = np.matrix([[1j, 2j], [3j, 4j]]) A.conj().transpose() |

inverse | inv(A) | solve(A) | np.linalg.inv(A) |

pseudoinverse | A = [0 1; 0 0] pinv(A) | install.packages('corpcor') library(corpcor) A = matrix(c(0, 0, 1, 0), nrow=2) pseudoinverse(A) | A = np.matrix([[0, 1], [0, 0]]) np.linalg.pinv(A) |

determinant | det(A) | det(A) | np.linalg.det(A) |

trace | trace(A) | sum(diag(A)) | A.trace() |

eigenvalues | eig(A) | eigen(A)$values | np.linalg.eigvals(A) |

eigenvectors | [evec, eval] = eig(A) evec(1:2) evec(3:4) | eigen(A)$vectors | np.linalg.eig(A)[1] |

singular value decomposition | X = randn(100) [u, d, v] = svd(X) | X = matrix(rnorm(100), nrow=10) result = svd(X) # singular values: result$d # matrix of eigenvectors: result$u # unitary matrix: result$v | |

solve system of equations | A \ [2;3] | solve(A, c(2, 3)) | np.linalg.solve(A, [2, 3]) |

sparse matrix construction | % 100x100 matrix with 5 at (1, 1) and 4 at (2, 2): X = sparse([1 2], [1 2], [5 4], 100, 100) | X = spMatrix(100, 100, c(1, 2), c(1, 2), c(5, 4)) | |

sparse matrix decomposition | [rows, cols, vals] = find(X) % just the values: nonzeros(X) | ||

sparse identity matrix | % 100x100 identity: speye(100) | ||

dense matrix to sparse matrixand back | X = sparse([1 0 0; 0 0 0; 0 0 0]) X2 = full(X) | ||

sparse matrix storage | % is storage sparse: issparse(X) % memory allocation in bytes: nzmax(X) % number of nonzero entries: nnz(X) | # memory allocation in bytes: object.size(X) | |

descriptive statistics | |||

matlab | r | numpy | |

first moment statistics | x = [1 2 3 8 12 19] sum(x) mean(x) | x = c(1,2,3,8,12,19) sum(x) mean(x) | x = [1,2,3,8,12,19] sp.sum(x) sp.mean(x) |

second moment statistics | std(x, 1) var(x, 1) | n = length(x) sd(x) * sqrt((n-1)/n) var(x) * (n-1)/n | sp.std(x) sp.var(x) |

second moment statistics for samples | std(x) var(x) | sd(x) var(x) | n = float(len(x)) sp.std(x) * math.sqrt(n/(n-1)) sp.var(x) * n/(n-1) |

skewness | Octave uses sample standard deviation to compute skewness:skewness(x) | install.packages('moments') library('moments') skewness(x) | stats.skew(x) |

kurtosis | Octave uses sample standard deviation to compute kurtosis:kurtosis(x) | install.packages('moments') library('moments') kurtosis(x) - 3 | stats.kurtosis(x) |

nth moment and nth central moment | n = 5 moment(x, n) moment(x, n, "c") | install.packages('moments') library('moments') n = 5 moment(x, n) moment(x, n, central=T) | n = 5??stats.moment(x, n) |

mode | mode([1 2 2 2 3 3 4]) | samp = c(1,2,2,2,3,3,4) names(sort(-table(samp)))[1] | stats.mode([1,2,2,2,3,3,4])[0][0] |

quantile statistics | min(x) median(x) max(x) iqr(x) ? | min(x) median(x) max(x) IQR(x) quantile(x, prob=.90) | min(x) sp.median(x) max(x) ??stats.scoreatpercentile(x, 90.0) |

bivariate statistiscscorrelation, covariance | x = [1 2 3] y = [2 4 7] cor(x, y) cov(x, y) | x = c(1,2,3) y = c(2,4,7) cor(x, y) cov(x, y) | x = [1,2,3] y = [2,4,7] stats.linregress(x, y)[2] ?? |

correlation matrix | x1 = randn(100, 1) x2 = 0.5 * x1 + randn(100, 1) x3 = 0.1 * x1 + 0.1 * x2 + 0.1 * randn(100, 1) corr([x1 x2 x3]) | x1 = rnorm(100) x2 = x1 + 0.5 * rnorm(100) x3 = 0.3 * x1 + 0.1 * 2 + 0.1 * rnorm(100) cor(cbind(x1, x2, x3)) | |

frequency table | x = c(1,2,1,1,2,5,1,2,7) tab = table(x) | ||

invert frequency table | rep(as.integer(names(tab)), unname(tab)) | ||

bin | x = c(1.1, 3.7, 8.9, 1.2, 1.9, 4.1) xf = cut(x, breaks=c(0, 3, 6, 9)) bins = tapply(x, xf, length) | ||

distributions | |||

matlab | r | numpy | |

binomialdensity, cumulative, quantile,sample of 10 | binopdf(x, n, p)binocdf(x, n, p)binoinv(y, n, p)binornd( n, p, 1, 10) | dbinom(x, n, p)pbinom(x, n, p)qbinom(y, n, p)rbinom(10, n, p) | stats.binom.pmf(x, n, p)stats.binom.cdf(x, n, p)stats.binom.ppf(y, n, p)stats.binom.rvs( n, p) |

poissondensity, cumulative, quantile,sample of 10 | poisspdf(x, lambda)poisscdf(x, lambda)poissinv(y, lambda)poissrnd( lambda, 1, 10) | dpois(x, lambda)ppois(x, lambda)qpois(y, lambda)rpois(10, lambda) | stats.poisson.pmf(x, lambda)stats.poisson.cdf(x, lambda)stats.poisson.ppf(y, lambda)stats.poisson.rvs( lambda, size=1) |

normaldensity, cumulative, quantile,sample of 10 | normpdf(x, mu, sigma)normcdf(x, mu, sigma)norminv(y, mu, sigma)normrnd( mu, sigma, 1, 10) | dnorm(x, mu, sigma)pnorm(x, mu, sigma)qnorm(y, mu, sigma)rnorm(10, mu, sigma) | stats.norm.pdf(x, mu, sigma)stats.norm.cdf(x, mu, sigma)stats.norm.ppf(y, mu, sigma)stats.norm.rvs( mu, sigma) |

gammadensity, cumulative, quantile,sample of 10 | gampdf(x, k, theta)gamcdf(x, k, theta)gaminv(y, k, theta)gamrnd( k, theta, 1, 10) | dgamma(x, k, scale=theta)pgamma(x, k, scale=theta)qgamma(y, k, scale=theta)rgamma(10, k, scale=theta) | stats.gamma.pdf(x, k, scale=theta)stats.gamma.cdf(x, k, scale=theta)stats.gamma.ppf(y, k, scale=theta)stats.gamma.rvs( k, scale=theta) |

exponentialdensity, cumulative, quantile,sample of 10 | exppdf(x, lambda)expcdf(x, lambda)expinv(y, lambda)exprnd( lambda, 1, 10) | dexp(x, lambda)pexp(x, lambda)qexp(y, lambda)rexp(10, lambda) | stats.expon.pdf(x, scale=1.0/lambda)stats.expon.cdf(x, scale=1.0/ lambda)stats.expon.ppf(x, scale=1.0/ lambda)stats.expon.rvs(scale=1.0/ lambda) |

chi-squareddensity, cumulative, quantile,sample of 10 | chi2pdf(x, nu)chi2cdf(x, nu)chi2inv(y, nu)chi2rnd( nu, 1, 10) | dchisq(x, nu)pchisq(x, nu)qchisq(y, nu)rchisq(10, nu) | stats.chi2.pdf(x, nu)stats.chi2.cdf(x, nu)stats.chi2.ppf(y, nu)stats.chi2.rvs( nu) |

betadensity, cumulative, quantile,sample of 10 | betapdf(x, alpha, beta)betacdf(x, alpha, beta)betainvf(y, alpha, beta)betarnd( alpha, beta, 1, 10) | dbeta(x, alpha, beta)pbeta(x, alpha, beta)qbeta(y, alpha, beta)rbeta(10, alpha, beta) | stats.beta.pdf(x, alpha, beta)stats.beta.cdf(x, alpha, beta)stats.beta.ppf(y, alpha, beta)stats.beta.pvs( alpha, beta) |

uniformdensity, cumulative, quantile,sample of 10 | unifpdf(x, a, b)unifcdf(x, a, b)unifinv(y, a, b)unifrnd( a, b, 1, 10) | dunif(x, a, b)punif(x, a, b)qunif(y, a, b)runif(10, a, b) | stats.uniform.pdf(x, a, b)stats.uniform.cdf(x, a, b)stats.uniform.ppf(y, a, b)stats.unifrom.rvs( a, b) |

Student's tdensity, cumulative, quantile,sample of 10 | tpdf(x, nu)tcdf(x, nu)tinv(y, nu)trnd( nu, 1, 10) | dt(x, nu)pt(x, nu)qt(y, nu)rt(10, nu) | stats.t.pdf(x, nu)stats.t.cdf(x, nu)stats.t.ppf(y, nu)stats.t.rvs( nu) |

Snedecor's Fdensity, cumulative, quantile,sample of 10 | fpdf(x, d1, d2)fcdf(x, d1, d2)finv(y, d1, d2)frnd( d1, d2, 1, 10) | df(x, d1, d2)pf(x, d1, d2)qf(y, d1, d2)rf(10, d1, d2) | stats.f.pdf(x, d1, d2)stats.f.cdf(x, d1, d2)stats.f.ppf(y, d1, d2)stats.f.rvs( d1, d2) |

empirical density function | % $ apt-get install octave-econometrics x = (-3:.05:3)' y = kernel_density(x, normrnd(0, 1, 100, 1)) | dfunc = density(rnorm(100)) dfunc$x dfunc$y | |

empirical cumulative distribution | F is a right-continuous step function:F = ecdf(rnorm(100)) | ||

empirical quantile function | F = ecdf(rnorm(100)) Finv = ecdf(F(seq(0, 1, .01))) | ||

linear models | |||

matlab | r | numpy | |

simple linear regressioncoefficient, intercept, and residuals | x = [1 2 3] y = [2 4 7] [lsq, res] = polyfit(x, y, 1) a = lsq(1) b = lsq(2) y - (a*x+b) | x = seq(10) y = 2 * x + 1 + rnorm(10) fit = lm(y ~ x) summary(fit) # yhat = ax + b: a = fit$coefficients[2] b = fit$coefficients[1] # y - (ax + b): fit$residuals | x = np.array([1,2,3]) y = np.array([2,4,7]) lsq = stats.linregress(x, y) a = lsq[0] b = lsq[1] y - (a*x+b) |

no intercept | x = seq(10) y = 2 * x + 1 + rnorm(10) fit = lm(y ~ x + 0) summary(fit) # y = ax: a = fit$coefficients[1] | ||

multiple linear regression | x1 = rnorm(100) x2 = rnorm(100) y = 2 * x2 + rnorm(100) fit = lm(y ~ x1 + x2) summary(fit) | ||

polynomial regression | x1 = rnorm(100) x2 = rnorm(100) y = 2 * x1 + x2 + 3 * x1 * x2 + rnorm(100) # x1, x2, and x1*x2 as predictors: fit = lm(y ~ x1 * x2) summary(fit) # just x1*x2 as predictor: fit2 = lm(Y ~ x1:x2) | ||

logistic regression | y = round(runif(100)) x1 = round(runif(100)) x2 = y + rnorm(100) fit = glm(y ~ x1 + x2, family="binomial") summary(fit) | ||

statistical tests | |||

matlab | r | numpy | |

wilcoxon signed-rank testvariable is symmetric around zero | x = unifrnd(-0.5, 0.5, 100, 1) % null hypothesis is true: wilcoxon_test(x, zeros(100, 1)) % alternative hypothesis is true: wilcoxon_test(x + 1.0, zeros(100, 1)) | # null hypothesis is true: wilcox.test(runif(100) - 0.5) alternative hypothesis is true: wilcox.test(runif(100) + 0.5) | stats.wilcoxon() |

kruskal-wallis rank sum testvariables have same location parameter | x = unifrnd(0, 1, 200, 1) % null hypothesis is true: kruskal_wallis_test(randn(100, 1), randn(200, 1)) % alternative hypothesis is true: kruskal_wallis_test(randn(100, 1), x) | # null hypothesis is true: kruskal.test(list(rnorm(100), rnorm(200))) # alternative hypothesis is true: kruskal.test(list(rnorm(100), runif(200))) | stats.kruskal() |

kolmogorov-smirnov testvariables have same distribution | x = randn(100, 1) y1 = randn(100, 1) y2 = unifrnd(-0.5, 0.5, 100, 1) % null hypothesis is true: kolmogorov_smirnov_test_2(x, y1) % alternative hypothesis is true: kolmogorov_smirnov_test_2(x, y2) | # null hypothesis is true: ks.test(rnorm(100), rnorm(100)) # alternative hypothesis is true: ks.test(rnorm(100), runif(100) - 0.5) | stats.ks_2samp() |

one-sample t-testmean of normal variable with unknown variance is zero | x1 = 3 * randn(100, 1) x2 = 3 * randn(100, 1) + 3 % null hypothesis is true: t_test(x1, 0) % alternative hypothesis is true: t_test(x2, 0) | # null hypothesis is true: t.test(rnorm(100, 0, 3)) # alternative hypothesis is true: t.test(rnorm(100, 3, 3)) | stats.ttest_1samp() |

independent two-sample t-testtwo normal variables have same mean | x = randn(100, 1) y1 = randn(100, 1) y2 = randn(100, 1) + 1.5 % null hypothesis is true: t_test_2(x, y1) % alternative hypothesis is true: t_test_2(x, y2) | # null hypothesis is true: t.test(rnorm(100), rnorm(100)) # alternative hypothesis is true: t.test(rnorm(100), rnorm(100, 3)) | stats.ttest_ind() |

one-sample binomial testbinomial variable parameter is as given | n = 100 x = rbinom(1, n, 0.5) # null hypothesis that p=0.5 is true: binom.test(x, n) # alternative hypothesis is true: binom.test(x, n, p=0.3) | stats.binom_test() | |

two-sample binomial testparameters of two binomial variables are equal | prop_test_2() | n = 100 x1 = rbinom(1, n, 0.5) x2 = rbinom(1, n, 0.5) # null hypothesis that p=0.5 is true: prop.test(c(x1, x2), c(n, n)) y = rbinom(1, n, 0.3) # alternative hypothesis is true: prop.test(c(x1, y), c(n, n)) | |

chi-squared testparameters of multinomial variable are all equal | chisquare_test_independence() | fair = floor(6 * runif(100)) + 1 loaded = floor(7 * runif(100)) + 1 loaded[which(loaded > 6)] = 6 # null hypothesis is true: chisq.test(table(fair)) # alternative hypothesis is true: chisq.test(table(loaded)) | stats.chisquare() |

poisson testparameter of poisson variable is as given | # null hypothesis is true: poisson.test(rpois(1, 100), r=100) # alternative test is true: poisson.test(rpois(1, 150), r=100) | ||

F testratio of variance of normal variables is as given | var_test() | x = rnorm(100) y = rnorm(100, 0, sd=sqrt(3)) # null hypothesis is true: var.test(y, x, ratio=3) # alternative hypothesis is true: var.test(y, x, ratio=1) | |

pearson product moment testnormal variables are not correlated | cor_test() | x1 = rnorm(100) x2 = rnorm(100) y = x2 + rnorm(100) # null hypothesis is true: cor.test(y, x1) # alternative hypothesis is true: cor.test(y, x2) | stats.pearsonr() |

shapiro-wilk testvariable has normal distribution | # null hypothesis is true: shapiro.test(rnorm(1000)) # alternative hypothesis is true: shapiro.test(runif(1000)) | stats.shapiro() | |

bartlett's testtwo or more normal variables have same variance | bartlett_test() | x = rnorm(100) y1 = rnorm(100) y2 = 0.1 * rnorm(100) # null hypothesis is true: bartlett.test(list(x, y1)) # alternative hypothesis is true: bartlett.test(list(x, y)) | stats.bartlett() |

levene's testtwo or more variables have same variance | install.packages('reshape', 'car') library(reshape) library(car) x = rnorm(100) y1 = rnorm(100) y2 = 0.1 * rnorm(100) # null hypothesis is true: df = melt(data.frame(x, y1)) leveneTest(df$value, df$variable) # alternative hypothesis is true: df = melt(data.frame(x, y2)) leveneTest(df$value, df$variable) | stats.levene() | |

one-way anovatwo or more normal variables have same mean | x1 = randn(100, 1) x2 = randn(100, 1) x3 = randn(100, 1) x = [x1; x2; x3] y = [x1; x2; x3 + 0.5] units = ones(100, 1) grp = [units; 2 * units; 3 * units] % null hypothesis is true: anova(x, grp) % alternative hypothesis is true: anova(y, grp) | install.packages('reshape') library(reshape) # null hypothesis that all means are the same # is true: x1 = rnorm(100) x2 = rnorm(100) x3 = rnorm(100) df = melt(data.frame(x1, x2, x3)) fit = lm(df$value ~ df$variable) anova(fit) | stats.f_oneway() |

time series | |||

matlab | r | numpy | |

time series | # first observation time is 1: y = ts(rnorm(100)) # first observation time is 0: y2 = ts(rnorm(100), start=0) plot(y) | # first observation time is 0: y = pd.Series(randn(100)) | |

multiple observations per time interval | # monthly observations 1993-1997: y = ts(rnorm(60), frequency=12) # monthly observations from Oct 1993: y2 = ts(rnorm(60), frequency=12, start=c(1993, 10)) plot(y) | ||

simple moving average | install.packages('TTR') library('TTR') ma = SMA(y, n=4) plot(y) lines(ma, col='red') | ||

weighted moving average | install.packages('TTR') library('TTR') ma = WMA(y, n=7, wts=c(1, 3, 5, 7, 5, 3, 1) plot(y) lines(ma, col='red') | ||

exponential smoothing | x = rnorm(100) fit = HoltWinters(x, beta=F, gamma=F) fit$a plot(fit) | ||

correlogram | x = rnorm(100) x2 = append(x[4:100], a[1:3]) acf(x, lag.max=20) acf(x + x2, lag.max=20) | ||

decompose into seasonal and trend | raw = seq(1,100) + rnorm(100) + rep(seq(1,10), 10) y = ts(raw, frequency=10) # additive model: t + s + r: yd = decompose(y) yd$trend yd$seasonal yd$random plot(yd) # multiplicative model: t * s * r: yd2 = decompose(y, type="multiplicative") | ||

arima | |||

curve fitting | |||

matlab | r | numpy | |

exact polynomial fit | x = [1 2 3 4] y = [3 9 2 1] % polynomial coefficient array: p = polyfit(x, y, 3) % plot polynomial: xx = -10:.1:10 yy = polyval(p, xx) plot(xx, yy) | ||

least square polynomial fit | |||

cubic spline | f = splinefun(rnorm(20)) x = seq(1, 20, .1) plot(x, f(x), type="l") | ||

univariate charts | |||

matlab | r | matplotlib | |

vertical bar chart | bar([7 3 8 5 5]) | cnts = c(7,3,8,5,5) names(cnts) = c("a","b","c","d","e") barplot(cnts) x = floor(6*runif(100)) barplot(table(x)) | cnts = [7,3,8,5,5] plt.bar(range(0,len(cnts)), cnts) |

horizontal bar chart | barh([7 3 8 5 5]) | cnts = c(7,3,8,5,5) names(cnts) = c("a","b","c","d","e") barplot(cnts, horiz=T) | cnts = [7,3,8,5,5] plt.barh(range(0,len(cnts)), cnts) |

pie chart | labels = {'a','b','c','d','e'} pie([7 3 8 5 5], labels) | cnts = c(7,3,8,5,5) names(cnts) = c("a","b","c","d","e") pie(cnts) | cnts = [7,3,8,5,5] labs = ['a','b','c','d','e'] plt.pie(cnts, labels=labs) |

stem-and-leaf plot | none | generates an ascii chart:stem(20*rnorm(100)) | none |

histogram | hist(randn(1, 100), 10) | hist(rnorm(100), breaks=10) | plt.hist(sp.randn(100), bins=range(-5,5)) |

box plot | boxplot(randn(1, 100)) boxplot([randn(1, 100) exprnd(1, 1, 100) unifrnd(0, 1, 1, 100)]') | boxplot(rnorm(100)) boxplot(rnorm(100), rexp(100), runif(100)) | plt.boxplot(sp.randn(100)) plt.boxplot([sp.randn(100), np.random.uniform(size=100), np.random.exponential(size=100)]) |

chart title | bar([7 3 8 5 5]) title('bar chart example') | all chart functions except for stem accept a main parameter:boxplot(rnorm(100), main="boxplot example", sub="to illustrate options") | plt.boxplot(sp.randn(100)) plt.title('boxplot example') |

grid of subplots | % 3rd arg refers to the subplot; % subplots are numbered in row-major order. for i = 1:4 subplot(2, 2, i), hist(randn(50)) end | for (i in split.screen(c(2, 2))) { screen(n=i) hist(rnorm(100)) } | |

open new plot window | open new plot figure open new plot | hist(rnorm(100)) dev.new() hist(rnorm(100)) | |

save plot as png | f = figure hist(randn(100)) print(f, '-dpng', 'histogram.png') | png('hist.png') hist(rnorm(100)) dev.off() | |

bivariate charts | |||

matlab | r | matplotlib | |

stacked bar chart | d = [7 1; 3 2; 8 1; 5 3; 5 1] bar(d, 'stacked') | d = matrix(c(7,1,3,2,8,1,5,3,5,1), nrow=2) labels = c("a","b","c","d","e") barplot(d,names.arg=labels) | a1 = [7,3,8,5,5] a2 = [1,2,1,3,1] plt.bar(range(0,5), a1, color='r') plt.bar(range(0,5), a2, color='b') |

grouped bar chart | d = [7 1; 3 2; 8 1; 5 3; 5 1] bar(d) | d = matrix(c(7,1,3,2,8,1,5,3,5,1), nrow=2) labels = c("a","b","c","d","e") barplot(d,names.arg=labels,beside=TRUE) | |

scatter plot | plot(randn(1,50),randn(1,50),'+') | plot(rnorm(50), rnorm(50)) | plt.scatter(sp.randn(50), sp.randn(50)) |

hexagonal binning | install.packages('hexbin') library('hexbin') plot(hexbin(rnorm(1000), rnorm(1000), xbins=12)) | hexbin(randn(1000), randn(1000), gridsize=12) | |

linear regression line | x = 0:20 y = 2 * x + rnorm(21)*10 fit = lm(y ~ x) plot(y) lines(x, fit$fitted.values, type='l') | x = range(0,20) err = sp.randn(20)*10 y = [2*i for i in x] + err A = np.vstack([x,np.ones(len(x))]).T m, c = np.linalg.lstsq(A, y)[0] plt.scatter(x, y) plt.plot(x, [m*i + c for i in x]) | |

polygonal line plot | plot(1:20,randn(1,20)) | plot(1:20, rnorm(20), type="l") | plot(range(0,20), randn(20)) |

function plot | fplot(@sin, [-4 4]) | x = seq(-4, 4, .01) plot(sin(x), type="l") | |

quantile-quantile plot | qqplot(runif(50),rnorm(50)) lines(c(-9,9), c(-9,9), col="red") | ||

axis labels | plot( 1:20, (1:20) .** 2) xlabel('x') ylabel('x squared') | plot(1:20, (1:20)^2, xlab="x", ylab="x squared") | |

axis limits | plot( 1:20, (1:20) .** 2) axis([1 20 -200 500]) | plot(1:20, (1:20)^2, xlim=c(0, 20), ylim=c(-200,500)) | |

logarithmic y-axis | semilogy(x, x .** 2, x, x .** 3, x, x .** 4, x, x .** 5) | x = 0:20 plot(x, x^2, log="y",type="l") lines(x, x^3, col="blue") lines(x, x^4, col="green") lines(x, x^5, col="red") | x = range(0, 20) for i in [2,3,4,5]: y.append([j**i for j in x]) for i in [0,1,2,3]: semilogy(x, y[i]) |

multivariate charts | |||

matlab | r | matplotlib | |

additional line set | plot(1:20, randn(1, 20), 1:20, randn(1, 20)) optional method:plot(1:20, randn(1, 20)) hold on plot(1:20, randn(1, 20)) | plot(1:20, rnorm(20), type="l") lines(1:20, rnorm(20), col="red") | |

superimposed plots with different y-axis scales | |||

legend | x = (1:20) y = x + rnorm(20) y2 = x - 2 + rnorm(20) plot(x, y, type="l", col="black") lines(x, y2, type="l", col="red") legend('topleft', c('first', 'second'), lty=c(1,1), lwd=c(2.5, 2.5), col=c('black', 'red')) | ||

additional point set | plot(rnorm(20), rnorm(20)) points(rnorm(20), rnorm(20), col='red') | ||

stacked area chart | x = rep(0:4, each=3) y = round(5 * runif(15)) letter = rep(LETTERS[1:3], 5) df = data.frame(x, y, letter) p = ggplot(df, aes(x=x, y=y, group=letter, fill=letter)) p + geom_area(position='stack') | ||

overlapping area chart | x = rep(0:4, each=3) y = round(5 * runif(15)) letter = rep(LETTERS[1:3], 5) df = data.frame(x, y, letter) alpha = rep(I(2/10), each=15) p = ggplot(df, aes(x=x, ymin=0, ymax=y, group=letter, fill=letter, alpha=alpha)) p + geom_ribbon() | ||

3d scatter plot | install.packages('scatterplot3d') library('scatterplot3d') scatterplot3d(rnorm(50), rnorm(50), rnorm(50), type="h") | ||

bubble chart | df = data.frame(x=rnorm(20), y=rnorm(20), z=rnorm(20)) p = ggplot(df, aes(x=x, y=y, size=z)) p + geom_point() | ||

scatter plot matrix | x = rnorm(20) y = rnorm(20) z = x + 3*y w = y + 0.1*rnorm(20) df = data.frame(x, y, z, w) pairs(df) | ||

contour plot | m = matrix(0, 100, 100) for (i in 2:100) { for (j in 2:100) { m[i,j] = (m[i-1,j] + m[i,j-1])/2 + runif(1) - 0.5 } } filled.contour(1:100, 1:100, m) | ||

_______________________________________________________ | _______________________________________________________ | _______________________________________________________ |

# General

## version used

The version of software used to check the examples in the reference sheet.

## show version

How to determine the version of an installation.

## implicit prologue

Code which examples in the sheet assume to have already been executed.

**r:**

The `ggplot2` library must be installed and loaded to use the plotting functions `qplot` and `ggplot`.

# Grammar and Invocation

## interpreter

How to invoke the interpreter on a script.

## repl

How to launch a command line read-eval-print loop for the language.

**r:**

R installations come with a GUI REPL.

The shell `zsh` has a built-in command `r` which re-runs the last command. Shell built-ins take precedence over external commands, but one can invoke the R REPL with:

`$ command r`

## command line program

How to pass the code to be executed to the interpreter as a command line argument.

## environment variables

How to get and set an environment variable.

## block delimiters

Punctuation or keywords which define blocks.

**matlab:**

The list of keywords which define blocks is not exhaustive. Blocks are also defined by

*switch*,*case*,*otherwise*,*endswitch**unwind_protect*,*unwind_protect_cleanup*,*end_unwind_protect**try*,*catch*,*end_try_catch*

## statement separator

How statements are separated.

**matlab:**

Semicolons are used at the end of lines to suppress output. Output echoes the assignment performed by a statement; if the statement is not an assignment the value of the statement is assigned to the special variable `ans`.

In Octave, but not MATLAB, newlines are not separators when preceded by a backslash.

## end-of-line comment

Character used to start a comment that goes to the end of the line.

**octave:**

Octave, but not MATLAB, also supports shell-style comments which start with `#`.

# Variables and Expressions

## assignment

**r:**

Traditionally <- was used in R for assignment. Using an = for assignment was introduced in version 1.4.0 sometime before 2002. -> can also be used for assignment:

`3 -> x`

## compound assignment

The compound assignment operators.

**octave:**

Octave, but not MATLAB, has compound assignment operators for arithmetic and bit operations:

```
+= -= *= /= **= ^=
&= |=
```

Octave, but not MATLAB, also has the C-stye increment and decrement operators `++` and `--`, which can be used in prefix and postfix position.

## increment and decrement operator

The operator for incrementing the value in a variable; the operator for decrementing the value in a variable.

## null

**matlab:**

`NaN` can be used for missing numerical values. Using a comparison operator on it always returns false, including `NaN == NaN`. Using a logical operator on `NaN` raises an error.

**octave:**

Octave, but not MATLAB, provides `NA` which is a synonym of `NaN`.

**r:**

Relational operators return `NA` when one of the arguments is `NA`. In particular `NA == NA` is `NA`. When acting on values that might be `NA`, the logical operators observe the rules of ternary logic, treating `NA` is the unknown value.

## null test

How to test if a value is null.

**octave:**

Octave, but not MATLAB, has `isna` and `isnull`, which are synonyms of `isnan` and `isempty`.

## conditional expression

A conditional expression.

# Arithmetic and Logic

## true and false

The boolean literals.

**matlab:**

*true* and *false* are functions which return matrices of ones and zeros of type *logical*. If no arguments are specified they return single entry matrices. If one argument is provided, a square matrix is returned. If two arguments are provided, they are the row and column dimensions.

## falsehoods

Values which evaluate to false in a conditional test.

**matlab:**

When used in a conditional, matrices evaluate to false unless they are nonempty and all their entries evaluate to true. Because strings are matrices of characters, an empty string ('' or "") will evaluate to false. Most other strings will evaluate to true, but it is possible to create a nonempty string which evaluates to false by inserting a null character; e.g. "false\000".

**r:**

When used in a conditional, a vector evaluates to the boolean value of its first entry. Using a vector with more than one entry in a conditional results in a warning message. Using an empty vector in a conditional, *c()* or *NULL*, raises an error.

## logical operators

The boolean operators.

**octave:**

Octave, but not MATLAB, also uses the exclamation point '!' for negation.

## relational operators

The relational operators.

**octave:**

Octave, but not MATLAB, also uses `!=` for an inequality test.

## arithmetic operators

The arithmetic operators: addition, subtraction, multiplication, division, quotient, remainder.

**matlab:**

*mod* is a function and not an infix operator. *mod* returns a positive value if the first argument is positive, whereas *rem* returns a negative value.

## integer division

How to compute the quotient of two integers.

## integer division by zero

What happens when an integer is divided by zero.

## float division

How to perform float division, even if the arguments are integers.

## float division by zero

What happens when a float is divided by zero.

## power

**octave:**

Octave, but not MATLAB, supports `**` as a synonym of `^`.

## sqrt

The square root function.

## sqrt(-1)

The result of taking the square root of a negative number.

## transcendental functions

The standard transcendental functions.

## transcendental constants

Constants for *pi* and *e*.

## float truncation

Ways of converting a float to a nearby integer.

## absolute value

The absolute value and signum of a number.

## integer overflow

What happens when an expression evaluates to an integer which is too big to be represented.

## float overflow

What happens when an expression evaluates to a float which is too big to be represented.

## float limits

The machine epsilon; the largest representable float and the smallest (i.e. closest to negative infinity) representable float.

## complex construction

Literals for complex numbers.

## complex decomposition

How to decompose a complex number into its real and imaginary parts; how to decompose a complex number into its absolute value and argument; how to get the complex conjugate.

## random number

How to generate a random integer from a uniform distribution; how to generate a random float from a uniform distribution.

## random seed

How to set, get, and restore the seed used by the random number generator.

**matlab:**

At startup the random number generator is seeded using operating system entropy.

**r:**

At startup the random number generator is seeded using the current time.

**numpy:**

On Unix the random number generator is seeded at startup from /dev/random.

## bit operators

The bit operators left shift, right shift, and, or , xor, and negation.

**matlab/octave:**

`bitshift` takes a second argument which is positive for left shift and negative for right shift.

`bitcmp` takes a second argument which is the size in bits of the integer being operated on. Octave is not compatible with MATLAB in how the integer size is indicated.

**r:**

There is a library on CRAN called `bitops` which provides bit operators.

# Strings

## literal

The syntax for a string literal.

## newline in literal

Can a newline be included in a string literal? Equivalently, can a string literal span more than one line of source code?

**octave:**

Double quote strings are Octave specific.

A newline can be inserted into a double quote string using the backslash escape `\n`.

A double quote string can be continued on the next line by ending the line with a backslash. No newline is inserted into the string.

## literal escapes

Escape sequences for including special characters in string literals.

**matlab:**

C-style backslash escapes are not recognized by string literals, but they are recognized by the IO system; the string 'foo\n' contains 5 characters, but emits 4 characters when written to standard output.

## concatenate

How to concatenate strings.

## replicate

How to create a string which consists of a character of substring repeated a fixed number of times.

## index of substring

How to get the index of first occurrence of a substring.

## extract substring

How to get the substring at a given index.

**octave:**

Octave supports indexing string literals directly: `'hello'(1:4)`.

## split

How to split a string into an array of substrings. In the original string the substrings must be separated by a character, string, or regex pattern which will not appear in the array of substrings.

The split operation can be used to extract the fields from a field delimited record of data.

**matlab:**

Cell arrays, which are essentially tuples, are used to store variable-length strings.

A two dimensional array of characters can be used to store strings of the same length, one per row. Regular arrays cannot otherwise be used to store strings.

## join

How to join an array of substrings into single string. The substrings can be separated by a specified character or string.

Joining is the inverse of splitting.

## trim

How to remove whitespace from the beginning and the end of a string.

Trimming is often performed on user provided input.

## pad

How to pad the edge of a string with spaces so that it is a prescribed length.

## number to string

How to convert a number to a string.

## string to number

How to convert a string to number.

## translate case

How to put a string into all caps. How to put a string into all lower case letters.

## sprintf

How to create a string using a printf style format.

## length

How to get the number of characters in a string.

## character access

How to get the character in a string at a given index.

**octave:**

Octave supports indexing string literals directly: `'hello'(1)`.

## chr and ord

How to convert an ASCII code to a character; how to convert a character to its ASCII code.

# Regular Expressions

## character class abbreviations

The supported character class abbreviations.

A character class is a set of one or more characters. In regular expressions, an arbitrary character class can be specified by listing the characters inside square brackets. If the first character is a circumflex `^`, the character class is all characters not in the list. A hyphen `-` can be used to list a range of characters.

**matlab:**

The C-style backslash escapes, which can be regarded as character classes which match a single character, are a feature of the regular expression engine and not string literals like in other languages.

## anchors

The supported anchors.

The `\<` and `\>` anchors match the start and end of a word respectively.

## match test

How to test whether a string matches a regular expression.

## case insensitive match test

How to perform a case insensitive match test.

## substitution

How to replace all substring which match a pattern with a specified string; how to replace the first substring which matches a pattern with a specified string.

## backreference in match and substitution

How to use backreferences in a regex; how to use backreferences in the replacement string of substitution.

# Date and Time

## current date/time

How to get the current date and time.

**r:**

`Sys.time()` returns a value of type `POSIXct`.

## date/time type

The data type used to hold a combined date and time value.

**matlab:**

The Gregorian calendar was introduced in 1582. The Proleptic Gregorian Calendar is sometimes used for earlier dates, but in the Proleptic Gregorian Calendar the year 1 CE is preceded by the year 1 BCE. The MATLAB epoch thus starts at the beginning of the year 1 BCE, but uses a zero to refer to this year.

## date/time difference type

The data type used to hold the difference between two date/time types.

## get date parts

How to get the year, the month as an integer from 1 through 12, and the day of the month from a date/time value.

**octave:**

In Octave, but not MATLAB, one can use index notation on the return value of a function:

```
t = now
datevec(t)(1)
```

## get time parts

How to get the hour as an integer from 0 through 23, the minute, and the second from a date/time value.

## build date/time from parts

How to build a date/time value from the year, month, day, hour, minute, and second as integers.

## convert to string

How to convert a date value to a string using the default format for the locale.

## strptime

How to parse a date/time value from a string in the manner of strptime from the C standard library.

## strftime

How to write a date/time value to a string in the manner of strftime from the C standard library.

# Tuples

## type

The name of the data type which implements tuples.

## literal

How to create a tuple, which we define as a fixed length, inhomogeneous list.

## lookup element

How to access an element of a tuple.

## update element

How to change one of a tuple's elements.

## length

How to get the number of elements in a tuple.

# Arrays

This section covers one-dimensional arrays which map integers to values.

Multidimensional arrays are a generalization which map tuples of integers to values.

Vectors and matrices are one-dimensional and two-dimensional arrays respectively containing numeric values. They support additional operations including the dot product, matrix multiplication, and norms.

Here are the data types covered in each section:

section | matlab | r | numpy |
---|---|---|---|

arrays | matrix (ndims = 1) | vector | list |

multidimensional arrays | matrix | array | np.array |

vectors | matrix (ndims = 1) | vector | np.array (ndim = 1) |

matrices | matrix (ndims = 2) | matrix | np.matrix |

## element type

How to get the type of the elements of an array.

## permitted element types

Permitted data types for array elements.

**matlab:**

Arrays in Octave can only contain numeric elements.

Array literals can have a nested structure, but Octave will flatten them. The following literals create the same array:

```
[ 1 2 3 [ 4 5 6] ]
[ 1 2 3 4 5 6 ]
```

Logical values can be put into an array because *true* and *false* are synonyms for 1 and 0. Thus the following literals create the same arrays:

```
[ true false false ]
[ 1 0 0 ]
```

If a string is encountered in an array literal, the string is treated as an array of ASCII values and it is concatenated with other ASCII values to produce as string. The following literals all create the same string:

```
[ 'foo', 98, 97, 114]
[ 'foo', 'bar' ]
'foobar'
```

If the other numeric values in an array literal that includes a string are not integer values that fit into a ASCII byte, then they are converted to byte sized values.

**r:**

Array literals can have a nested structure, but R will flatten them. The following literals produce the same array of 6 elements:

```
c(1,2,3,c(4,5,6))
c(1,2,3,4,5,6)
```

If an array literal contains a mixture of booleans and numbers, then the boolean literals will be converted to 1 (for TRUE and T) and 0 (for FALSE and F).

If an array literal contains strings and either booleans or numbers, then the booleans and numbers will be converted to their string representations. For the booleans the string representations are "TRUE'" and "FALSE".

## literal

The syntax, if any, for an array literal.

**matlab:**

The array literal

`[1,'foo',3]`

will create an array with 5 elements of class

*char*.

**r:**

The array literal

`c(1,'foo',3)`

will create an array of 3 elements of class *character*, which is the R string type.

## size

How to get the number of values in an array.

## empty test

## lookup

## update

## out-of-bounds behavior

## index of element

## slice

## slice to end

## concatenate

## replicate

## copy

How to make an address copy, a shallow copy, and a deep copy of an array.

After an address copy is made, modifications to the copy also modify the original array.

After a shallow copy is made, the addition, removal, or replacement of elements in the copy does not modify of the original array. However, if elements in the copy are modified, those elements are also modified in the original array.

A deep copy is a recursive copy. The original array is copied and a deep copy is performed on all elements of the array. No change to the contents of the copy will modify the contents of the original array.

# Arithmetic Sequences

An arithmetic sequence is a sequence of numeric values in which consecutive terms have a constant difference.

## unit difference

An arithmetic sequence with a difference of 1.

## difference of 10

An arithmetic sequence with a difference of 10.

## difference of 0.1

An arithmetic sequence with a difference of 0.1.

## iterate

How to iterate over an arithmetic sequence.

## convert to array

How to convert an arithmetic sequence to an array.

# Multidimensional Arrays

Multidimensional arrays are a generalization of arrays which map tuples of integers to values. All tuples in the domain of a multidimensional array have the same length; this length is the dimension of the array.

The multidimensional arrays described in this sheet are homogeneous, meaning that the values are all of the same type. This restriction allows the implementation to store the values of the multidimensional array in a contiguous region of memory without the use of references or points.

Multidimensional arrays should be contrasted with nested arrays. When arrays are nested, the innermost nested arrays contain the values and the other arrays contain references to arrays. The syntax for looking up a value is usually different:

```
# nested:
a[1][2]
# multidimensional:
a[1, 2]
```

## element type

How to get the type of the values stored in a multidimensional array.

**r:**

## literal—1d

## literal—2d

## construct from sequence—2d

## construct from sequence—3d

## construct from nested sequences—2d

## construct from nested sequences—3d

## construct from rows—2d

## construct from columns—2d

## construct from subarrays—2d

## construct 3d array from 2d arrays

## size

## lookup—1d

## lookup—2d

## 1d lookup on 2d array

## update—2d

## out-of-bounds behavior

## slice

## slice to end

## slice subarray

## transpose

## swap axes

## flip—2d

## flip—3d

## circular shift—2d

## rotate—2d

## apply function element-wise

## apply function to linear subarrays

# Dictionaries

## literal

The syntax for a dictionary literal.

## size

How to get the number of keys in a dictionary.

## lookup

How to use a key to lookup a value in a dictionary.

## update

How to add or key-value pair or change the value for an existing key.

## missing key behavior

What happens when looking up a key that isn't in the dictionary.

## delete

How to delete a key-value pair from a dictionary.

## iterate

How to iterate over the key-value pairs.

## keys and values as arrays

How to get an array containing the keys; how to get an array containing the values.

## merge

How to merge two dictionaries.

# Tables

Tables are a data type which correspond to the tables of relational databases. In R this data type is called a *data frame*. The Python library Pandas provides a table data type called *DataFrame*.

A table is an array of tuples, each of the same length and type. If the type of the first element of the first type is integer, then all the tuples in the table must have first elements which are integers. The type of the tuples corresponds to the schema of a relational database table.

A table can also be

Pandas types: Series(), DataFrame(), Index()

## construct from column arrays

How to construct a data frame from a set of arrays representing the columns.

**octave:**

Octave does not have the `table` data type.

## size

How to get the number of columns and number of rows in a table.

## construct from row tuples

## column names as array

How to show the names of the columns.

## access column as array

How to access a column in a data frame.

## access row as tuple

How to access a row in a data frame.

**r:**

*people[1, ]* returns the 1st row from the data frame *people* as a new data frame with one row. This can be converted to a list using the function *as.list*. There is often no need because lists and one row data frames have nearly the same behavior.

## access datum

How to access a single datum in a data frame; i.e. the value in a column of a single row.

## order rows by column

How to sort the rows in a data frame according to the values in a specified column.

## order rows by multiple columns

## order rows in descending order

How to sort the rows in descending order according to the values in a specified column.

## limit rows

How to select the first *n* rows according to some ordering.

## offset rows

How to select rows starting at offset *n* according to some ordering.

## attach columns

How to make column name a variable in the current scope which refers to the column as an array.

**r:**

Each column of the data frame is copies into a variable named after the column containing the column as a vector. Modifying the data in the variable does not alter the original data frame.

## detach columns

How to remove attached column names from the current scope.

## spreadsheet editor

How to view and edit the data frame in a spreadsheet.

# Import and Export

## import tab delimited file

Load a data frame from a tab delimited file.

## import comma-separated values file

Load a data frame from a CSV file.

## set column separator

How to set the column separator when importing a delimited file.

## set quote character

How to change the quote character. Quoting is used when strings contain the column separator or the line terminator.

## import file w/o header

How to import a file that lacks a header.

## set column names

How to set the column names.

## set column types

How to indicate the type of the columns.

**r:**

If the column types are not set or if the type is set to NA or NULL, then the type will be set to logical, integer, numeric, complex, or factor.

## recognize null values

Specify the input values which should be converted to null values.

## unequal row length behavior

What happen when a row of input has less than or more than the expected number of columns.

## skip comment lines

How to skip comment lines.

## skip rows

## maximum rows to read

## index column

## export tab delimited file

## export comma-separated values file

Save a data frame to a CSV file.

**r:**

If row.names is not set to F, the initial column will be the row number as a string starting from "1".

# Relational Algebra

## map data frame

How to apply a mapping transformation to the rows of a data set.

## filter data set

How to select the rows of a data set that satisfy a predicate.

# Aggregation

# Functions

## define function

How to define a function.

## invoke function

How to invoke a function.

## nested function

## missing argument behavior

What happens when a function is invoked with too few arguments.

## extra argument behavior

What happens when a function is invoked with too many arguments.

## default argument

How to assign a default argument to a parameter.

## variadic function

How to define a function which accepts a variable number of arguments.

## return value

How the return value of a function is determined.

## multiple return values

How to return multiple values from a function.

## anonymous function literal

The syntax for an anonymous function.

## invoke anonymous function

## closure

## function as value

How to store a function in a variable.

# Execution Control

## if

How to write a branch statement.

## while

How to write a conditional loop.

## for

How to write a C-style for statement.

## break/continue

How to break out of a loop. How to jump to the next iteration of a loop.

## raise exception

How to raise an exception.

## handle exception

How to handle an exception.

# File Handles

## standard file handles

Standard input, standard output, and standard error.

## read line from stdin

## write line to stdout

How to write a line to stdout.

**matlab:**

The backslash escape sequence `\n` is stored as two characters in the string and interpreted as a newline by the IO system.

## write formatted string to stdout

## open file for reading

## open file for writing

## open file for appending

## close file

## i/o errors

## read line

## iterate over file by line

## read file into array of strings

## write string

## write line

## flush file handle

## file handle position

## redirect stdout to file

# Directories

## working directory

How to get and set the working directory.

# Processes and Environment

## command line arguments

How to get the command line arguments.

## environment variables

How to get and set and environment variable.

# Libraries and Namespaces

## load library

How to load a library.

## list loaded libraries

Show the list of libraries which have been loaded.

## library search path

The list of directories the interpreter will search looking for a library to load.

## source file

How to source a file.

**r:**

When sourcing a file, the suffix if any must be specified, unlike when loading library. Also, a library may contain a shared object, but a sourced file must consist of just R source code.

## install package

How to install a package.

## list installed packages

How to list the packages which have been installed.

# Reflection

## data type

How to get the data type of a value.

**r:**

For vectors `class` returns the *mode* of the vector which is the type of data contained in it. The possible modes are

- numeric
- complex
- logical
- character
- raw

Some of the more common class types for non-vector entities are:

- matrix
- array
- list
- factor
- data.frame

## attributes

How to get the attributes for an object.

**r:**

Arrays and vectors do not have attributes.

## methods

How to get the methods for an object.

## variables in scope

How to list the variables in scope.

## undefine variable

How to undefine a variable.

## undefine all variables

How to undefine all variables.

## eval

How to interpret a string as source code and execute it.

## function documentation

How to get the documentation for a function.

## list library functions

How to list the functions and other definitions in a library.

## search documentation

How to search the documentation by keyword.

# Vectors

A vector is a one dimensional array which supports these operations:

- addition on vectors of the same length
- scalar multiplication
- a dot product
- a norm

The languages in this reference sheet provide the above operations for all one dimensional arrays which contain numeric values.

## vector literal

## element-wise arithmetic operators

## scalar multiplication

## dot product

## cross product

## norms

**matlab:**

The *norm* function returns the p-norm, where the second argument is *p*. If no second argument is provided, the 2-norm is returned.

# Matrices

## literal or constructor

Literal syntax or constructor for creating a matrix.

The elements of a matrix must be specified in a linear order. If the elements of each row of the matrix are adjacent to other elements of the same row in the linear order we say the order is *row-major*. If the elements of each column are adjacent to other elements of the same column we say the order is *column-major*.

**matlab:**

Square brackets are used for matrix literals. Semicolons are used to separate rows, and commas separate row elements. Optionally, newlines can be used to separate rows and whitespace to separate row elements.

**r:**

Matrices are created by passing a vector containing all of the elements, as well as the number of rows and columns, to the *matrix* constructor.

If there are not enough elements in the data vector, the values will be recycled. If there are too many extra values will be ignored. However, the number of elements in the data vector must be a factor or a multiple of the number of elements in the final matrix or an error results.

When consuming the elements in the data vector, R will normally fill by column. To change this behavior pass a *byrow=T* argument to the *matrix* constructor:

`A = matrix(c(1,2,3,4),nrow=2,byrow=T)`

## constant matrices

How to create a matrices with zeros for entries or with ones for entries.

## diagonal matrices

How to create diagonal matrices including the identity matrix.

A matrix is diagonal if and only if `a _{ij} = 0` for all

`i ≠ j`.

## dimensions

How to get the dimensions of a matrix.

## element access

How to access an element of a matrix. All languages described here follow the convention from mathematics of specifying the row index before the column index.

**matlab:**

Rows and columns are indexed from one.

**r:**

Rows and columns are indexed from one.

## row access

How to access a row.

## column access

How to access a column.

## submatrix access

How to access a submatrix.

## scalar multiplication

How to multiply a matrix by a scalar.

## element-wise operators

Operators which act on two identically sized matrices element by element. Note that element-wise multiplication of two matrices is used less frequently in mathematics than matrix multiplication.

```
from numpy import array
matrix(array(A) * array(B))
matrix(array(A) / array(B))
```

## multiplication

How to multiply matrices. Matrix multiplication should not be confused with element-wise multiplication of matrices. Matrix multiplication in non-commutative and only requires that the number of columns of the matrix on the left match the number of rows of the matrix. Element-wise multiplication, by contrast, is commutative and requires that the dimensions of the two matrices be equal.

## kronecker product

The Kronecker product is a non-commutative operation defined on any two matrices. If A is m x n and B is p x q, then the Kronecker product is a matrix with dimensions mp x nq.

## comparison

How to test two matrices for equality.

**matlab:**

== and != perform entry-wise comparison. The result of using either operator on two matrices is a matrix of boolean values.

~= is a synonym for !=.

**r:**

== and != perform entry-wise comparison. The result of using either operator on two matrices is a matrix of boolean values.

## norms

How to compute the 1-norm, the 2-norm, the infinity norm, and the frobenius norm.

**matlab:**

*norm(A)* is the same as *norm(A,2)*.

## sparse matrix construction

How to construct a sparse matrix using coordinate format.

Coordinate format specifies a matrix with three arrays: the row indices, the the column indices, and the values.

## sparse matrix decomposition

## sparse identity matrix

## dense matrix to sparse matrix

## sparse matrix storage

# Descriptive Statistics

A statistic is a single number which summarizes a population of data. The most familiar example is the mean or average. Statistics defined for discrete populations can often be meaningfully extended to continuous distributions by replacing summations with integration.

An important class of statistics are the nth moments. The nth moment
$$\mu'_n$$
of a population of *k* values *x _{i}* with mean

*μ*is:

The nth central moment *μ _{n}* of the same population is:

## first moment statistics

The sum and the mean.

The mean is the first moment. It is one definition of the center of the population. The median and the mode are also used to define the center. In most populations they will be close to but not identical to the mean.

## second moment statistics

The variance and the standard deviation. The variance is the second central moment. It is a measure of the spread or width of the population.

The standard deviation is the square root of the variance. It is also a measurement of population spread. The standard deviation has the same units of measurement as the data in the population.

## second moment statistics for samples

The sample variance and sample standard deviation.

## skewness

The skewness of a population.

The skewness measures the asymmetrically of the population. The skewness will be negative, positive, or zero when the population is more spread out on the left, more spread out on the right, or similarly spread out on both sides, respectively.

The skewness can be calculated from the third moment and the standard deviation:

When estimating the population skewness from a sample a correction factor is often used, yielding the sample skewness:

**octave and matlab:**

Octave uses the sample standard deviation to compute skewness. This behavior is different from Matlab and should possibly be regarded as a bug.

Matlab, but not Octave, will take a flag as a second parameter. When set to zero Matlab returns the sample skewness:

`skewness(x, 0)`

**numpy:**

Set the named parameter `bias` to `False` to get the sample skewness:

`stats.skew(x, bias=False)`

## kurtosis

The kurtosis of a population.

The formula for kurtosis is:

When kurtosis is negative the sides of a distribution tend to be more convex than when the kurtosis is is positive. A negative kurtosis distribution tends to have a wide, flat peak and narrow tails. Such a distribution is called platykurtic. A positive kurtosis distribution tends to have a narrow, sharp peak and long tails. Such a distribution is called leptokurtic.

The fourth standardized moment is

The fourth standardized moment is sometimes taken as the definition of kurtosis in older literature. The reason the modern definition is preferred is because it assigns the normal distribution a kurtosis of zero.

**matlab:**

Octave uses the sample standard deviation when computing kurtosis. This should probably be regarded as a bug.

**r:**

R uses the older fourth standardized moment definition of kurtosis.

## nth moment and nth central moment

How to compute the nth moment (also called the nth absolute moment) and the nth central moment for arbitrary *n*.

## mode

The mode is the most common value in the sample.

The mode is a measure of central tendency like the mean and the median. A problem with the mean is that it can produce values not found in the data. For example the mean number of persons in an American household was 2.6 in 2009.

The mode might not be unique. If there are two modes the sample is said to be bimodal, and in general if there is more than one mode the sample is said to be multimodal.

## quantile statistics

If the data is sorted from smallest to largest, the *minimum* is the first value, the *median* is the middle value, and the *maximum* is the last value. If there are an even number of data points, the median is the average of the two middle points. The median divides the population into two halves.

When the population is divided into four parts the division markers are called the first, second, and third *quartiles*. The *interquartile range* (IQR) is the difference between the 3rd and 1st quartiles.

When the population is divided into ten parts the division markers are called *deciles*.

When the population is divided into a hundred parts the division markers are called *percentiles*.

If the population is divided into *n* parts the markers are called the 1st, 2nd, …, (n-1)th n-*quantiles*.

## bivariate statistics

The correlation and the covariance.

The correlation is a number from -1 to 1. It is a measure of the linearity of the data, with values of -1 and 1 representing indicating a perfectly linear relationship. When the correlation is positive the quantities tend to increase together and when the correlation is negative one quantity will tend to increase as the other decreases.

A variable can be completely dependent on another and yet the two variables can have zero correlation. This happens for Y = X^{2} where uniform X on the interval [-1, 1]. Anscombe's quartet gives four examples of data sets each with the same fairly high correlation 0.816 and yet which show significant qualitative differences when plotted.

The covariance is defined by

The correlation is the normalized version of the covariance. It is defined by

## correlation matrix

## frequency table

How to compute the frequency table for a data set. A frequency table counts how often each value occurs in the data set.

**r:**

The `table` function returns an object of type `table`.

## invert frequency table

How to convert a frequency table back into the original data set.

The order of the original data set is not preserved.

## bin

How to bin a data set. The result is a frequency table where each frequency represents the number of samples from the data set for an interval.

**r:**

The `cut` function returns a `factor`.

A `labels` parameter can be provided with a vector argument to assign the bins names. Otherwise bin names are constructed from the breaks using "[0.0,1.0)" style notation.

The `hist` function can be used to bin a data set:

```
x = c(1.1, 3.7, 8.9, 1.2, 1.9, 4.1)
hist(x, breaks=c(0, 3, 6, 9), plot=FALSE)
```

`hist` returns an object of type `histogram`. The counts are in the `$counts` attribute.

# Distributions

A distribution density function *f(x)* is a non-negative function which, when integrated over its entire domain is equal to one. The distributions described in this sheet have as their domain the real numbers. The support of a distribution is the part of the domain on which the density function is non-zero.

A distribution density function can be used to describe the values one is likely to see when drawing an example from a population. Values in areas where the density function is large are more likely than values in areas where the density function is small. Values where there density function is zero do not occur. Thus it can be useful to plot the density function.

To derive probabilities from a density function one must integrate or use the associated cumulative density function

which gives the probability of seeing a value less than or equal to *x*. As probabilities are non-negative and no greater than one, *F* is a function from (-∞, ∞) to [0,1]. The inverse of F is called the inverse cumulative distribution function or the quantile function for the distribution.

For each distribution statistical software will generally provide four functions: the density, the cumulative distribution, the quantile, and a function which returns random numbers in frequencies that match the distribution. If the software does not provide a random number generating function for the distribution, the quantile function can be composed with the built-in random number generator that most languages have as long as it returns uniformly distributed floats from the interval [0, 1].

density probability density probability mass | cumulative density cumulative distribution distribution | inverse cumulative density inverse cumulative distribution quantile percentile percent point | random variate |

Discrete distributions such as the binomial and the poisson do not have density functions in the normal sense. Instead they have probability mass functions which assign probabilities which sum up to one to the integers. In R warnings will be given if non integer values are provided to the mass functions `dbinom` and `dpoiss`.

The cumulative distribution function of a discrete distribution can still be defined on the reals. Such a function is constant except at the integers where it may have jump discontinuities.

Most well known distributions are in fact parametrized families of distributions. This table lists some of them with their parameters and properties.

The information entropy of a continuous distribution with density *f(x)* is defined as:

In Bayesian analysis the distribution with the greatest entropy, subject to the known facts about the distribution, is called the maximum entropy probability distribution. It is considered the best distribution for modeling the current state of knowledge.

## binomial

The probability mass, cumulative distribution, quantile, and random number generating functions for the binomial distribution.

The binomial distribution is a discrete distribution. It models the number of successful trails when *n* is the number of trials and *p* is the chance of success for each trial. An example is the number of heads when flipping a coin 100 times. If the coin is fair then *p* is 0.50.

**numpy:**

Random numbers in a binomial distribution can also be generated with:

`np.random.binomial(n, p)`

## poisson

The probability mass, cumulative distribution, quantile, and random number generating functions for the Poisson distribution.

The Poisson distribution is a discrete distribution. It is described by a parameter *lam* which is the mean value for the distribution. The Poisson distribution is used to model events which happen at a specified average rate and independently of each other. Under these circumstances the time between successive events will be described by an exponential distribution and the events are said to be described by a poisson process.

**numpy:**

Random numbers in a Poisson distribution can also be generated with:

`np.random.poisson(lam, size=1)`

## normal

The probability density, cumulative distribution, quantile, and random number generating functions for the normal distribution.

The parameters are the mean μ and the standard deviation σ. The standard normal distribution has μ of 0 and σ of 1.

The normal distribution is the maximum entropy distribution for a given mean and variance. According to the central limit theorem, if {X_{1}, …, X_{n}} are any independent and identically distributed random variables with mean μ and variance σ^{2}, then S_{n} := Σ X_{i} / n converges to a normal distribution with mean μ and variance σ^{2}/n.

**numpy:**

Random numbers in a normal distribution can also be generated with:

`np.random.randn()`

## gamma

The probability density, cumulative distribution, quantile, and random number generating functions for the gamma distribution.

The parameter *k* is called the shape parameter and θ is called the scale parameter. The rate of the distribution is β = 1/θ.

If X_{i} are *n* independent random variables with Γ(k_{i}, θ) distribution, then Σ X_{i} has distribution Γ(Σ k_{i}, θ).

If X has Γ(k, θ) distribution, then αX has Γ(k, αθ) distribution.

## exponential

The probability density, cumulative distribution, quantile, and random number generating functions for the exponential distribution.

## chi-squared

The probability density, cumulative distribution, quantile, and random number generating functions for the chi-squared distribution.

## beta

The probability density, cumulative distribution, quantile, and random number generating functions for the beta distribution.

## uniform

The probability density, cumulative distribution, quantile, and random number generating functions for the uniform distribution.

The uniform distribution is described by the parameters *a* and *b* which delimit the interval on which the density function is nonzero.

The uniform distribution the is maximum entropy probability distribution with support *[a, b]*.

Consider the uniform distribution on *[0, b]*. Suppose that we take *k* samples from it, and *m* is the largest of the samples. The minimum variance unbiased estimator for *b* is

**octave, r, numpy:**

*a* and *b* are optional parameters and default to 0 and 1 respectively.

## Student's t

The probability density, cumulative distribution, quantile, and random number generating functions for Student's t distribution.

## Snedecor's F

The probability density, cumulative distribution, quantile, and random number generating functions for Snedecor's F distribution.

## empirical density function

How to construct a density function from a sample.

## empirical cumulative distribution

## empirical quantile function

# Linear Models

## simple linear regression

How to get the slope *a* and intercept *b* for a line which best approximates the data. How to get the residuals.

If there are more than two data points, then the system is overdetermined and in general there is no solution for the slope and the intercept. Linear regression looks for line that fits the points as best as possible. The least squares solution is the line that minimizes the sum of the square of the distances of the points from the line.

The residuals are the difference between the actual values of *y* and the calculated values using *ax + b*. The norm of the residuals can be used as a measure of the goodness of fit.

## no intercept

## multiple linear regression

## polynomial regression

## logistic regression

# Statistical Tests

A selection of statistical tests. For each test the null hypothesis of the test is stated in the left column.

In a null hypothesis test one considers the *p-value*, which is the chance of getting data which is as or more extreme than the observed data if the null hypothesis is true. The null hypothesis is usually a supposition that the data is drawn from a distribution with certain parameters.

The extremeness of the data is determined by comparing the expected value of a parameter according to the null hypothesis to the estimated value from the data. Usually the parameter is a mean or variance. In a *one-tailed test* the p-value is the chance the difference is greater than the observed amount; in a *two-tailed test* the p-value is the chance the absolute value of the difference is greater than the observed amount.

Octave and MATLAB have different names for the statistical test functions. The sheet shows the Octave functions; the corresponding MATLAB functions are:

octave | matlab |
---|---|

wilcoxon_test | ranksum |

kruskal_wallis_test | kruskalwallis |

kolmogorov_smirnov_test | kstest |

kolmogorov_smirnov_test_2 | kstest2 |

t_test | ttest |

t_test_2 | ttest2 |

## wilcoxon signed-rank test

**matlab**

`wilcoxon_test()` is an Octave function. The MATLAB function is `ranksum()`.

## kruskal-wallis rank sum test

## kolmogorov-smirnov test

Test whether two samples are drawn from the same distribution.

**matlab:**

`kolmogorov_smirnov_test_2()` and `kolmogorov_smirnov_test()` are Octave functions. The corresponding MATLAB functions are `kstest2()` and `kstest()`.

`kolmogorov_smirnov_test()` is a one sample test; it tests whether a sample is drawn from one of the standard continuous distributions. A one sample KS test gives a repeatable p-value; generating a sample and using a two sample KS test does not.

```
x = randn(100, 1)
% null hypothesis is true:
kolmogorov_smirnov_test(x, "norm", 0, 1)
% alternative hypothesis is true:
kolmogorov_smirnov_test(x, "unif", -0.5, 0.5)
```

**r:**

## one-sample t-test

## independent two-sample t-test

Test whether two normal variables have same mean.

**r:**

If the normal variables are known to have the same variance, the variance can be pooled to estimate standard error:

`t.test(x, y, var.equal=T)`

If the variance cannot be pooled, then Welch's t-test is used. This uses a lower (often non-integral) degrees-of-freedom value, which in turn results in a higher p-value.

## one-sample binomial test

## two-sample binomial test

## chi-squared test

## poisson test

## F test

## pearson product moment test

## shapiro-wilk test

## bartlett's test

A test whether variables are drawn from normal distributions with the same variance.

## levene's test

A test whether variables are drawn from distributions with the same variance.

## one-way anova

# Time Series

## time series

## multiple observations per time interval

## simple moving average

## weighted moving average

## exponential smoothing

## correlogram

## decompose into seasonal and trend

## arima

# Curve Fitting

## exact polynomial fit

## least squares polynomial fit

## cubic spline

How to connect the dots of a data set with a line which has a continuous 2nd derivative.

# Univariate Charts

## vertical bar chart

A chart in which numerical values are represented by horizontal bars. The bars are aligned at the bottom.

**r:**

How to produce a bar chart using ggplot2:

```
cnts = c(7,3,8,5,5)
names = c("a","b","c","d","e")
df = data.frame(names, cnts)
qplot(names, data=df, geom="bar", weight=cnts)
```

## horizontal bar chart

A bar chart with horizontal bars which are aligned on the left.

## pie chart

A pie chart displays values using the areas of circular sectors or equivalently the lengths of the arcs of those sectors.

A pie chart implies that the values are percentages of a whole.

## stem-and-leaf plot

A stem-and-leaf plot is a concise way of storing a small set of numbers which makes their distribution visually evident.

The original set of numbers can be recovered with some loss of accuracy by appending the number on the left with each of the digits on the right. In the example below the original data set contained -43, -42, -41, -39, -38, -35, …, 35, 44, 46, 50, 58.

```
> stem(20*rnorm(100))
The decimal point is 1 digit(s) to the right of the |
-4 | 321
-2 | 98544054310
-0 | 8864333111009998776444332222110
0 | 0001122333333466667778899122334555666789
2 | 00023669025
4 | 4608
```

## histogram

A histogram is a bar chart where each bar represents a range of values that the data points can fall in. The data is tabulated to find out how often data points fall in each of the bins and in the final chart the length of the bars corresponds to the frequency.

A common method for choosing the number of bins using the number of data points is Sturges' formula:

**r:**

How to make a histogram with the `ggplot2` library:

```
qplot(rnorm(50), geom="histogram", binwidth=binwidth)
binwidth = (max(x)-min(x))/10
qplot(rnorm(50), geom="histogram", binwidth=binwidth)
```

## box plot

Also called a box-and-whisker plot.

The box shows the locations of the 1st quartile, median, and 3rd quartile. These are the same as the 25th percentile, 50th percentile, and 75th percentile.

The whiskers are sometimes used to show the maximum and minimum values of the data set. Outliers are sometimes shown explicitly with dots, in which case all remaining data points occur inside the whiskers.

**r:**

How to create a box plot with `ggplot2`:

```
qplot(x="rnorm", y=rnorm(50), geom="boxplot")
qplot(x=c("rnorm", "rexp", "runif"), y=c(rnorm(50), rexp(50), runif(50)), geom="boxplot")
```

## chart title

How to set the chart title.

**r:**

The `qplot` commands supports the `main` options for setting the title:

`qplot(x="rnorm", y=rnorm(50), geom="boxplot", main="boxplot example")`

# Bivariate Charts

## stacked bar chart

Two or more data sets with a common set of labels can be charted with a stacked bar chart. This makes the sum of the data sets for each label readily apparent.

## grouped bar chart

Optionally data sets with a common set of labels can be charted with a grouped bar chart which clusters the bars for each label. The grouped bar chart makes it easier to perform comparisons between labels for each data set.

## scatter plot

A scatter plot can be used to determine if two variables are correlated.

**r:**

How to make a scatter plot with `ggplot`:

```
x = rnorm(50)
y = rnorm(50)
p = ggplot(data.frame(x, y), aes(x, y))
p = p + layer(geom="point")
p
```

## hexagonal binning

A hexagonal binning is the two-dimensional analog of a histogram. The number of data points in each hexagon is tabulated, and then color or grayscale is used to show the frequency.

A hexagonal binning is superior to a scatter-plot when the number of data points is high because most scatter-plot software doesn't indicate when points are occur on top of each other.

## linear regression line

How to plot a line determined by linear regression on top of a scatter plot.

## polygonal line plot

How to connect the dots of a data set with a polygonal line.

## function plot

How to plot a function.

## quantile-quantile plot

Also called a Q-Q plot.

A quantile-quantile plot is a scatter plot created from two data sets. Each point depicts the quantile of the first data set with its x position and the corresponding quantile of the second data set with its y position.

If the data sets are drawn from the same distribution then most of the points should be close to the line y = x. If the data sets are drawn from distributions which have a linear relation then the Q-Q plot should also be close to linear.

## axis labels

How to label the x and y axes.

**r:**

How to label the axes with ggplot2:

```
x = rnorm(20)
y = x^2
p = ggplot(data.frame(x, y), aes(x, y))
p + layer(geom="point") + xlab('x') + ylab('x squared')
```

## axis limits

How to manually set the range of values displayed by an axis.

## logarithmic y-axis

# Multivariate Charts

## additional line set

## legend

How to put a legend on a chart.

**r:**

These strings can be used as the first argument to control the legend position:

- "bottomright"
- "bottom"
- "bottomleft"
- "left"
- "topleft"
- "top"
- "topright"
- "right"
- "center"

The named parameter `lwd` is the line width. It is roughly the width in pixels, though the exact interpretation is device specific.

The named parameter `lty` specifies the line type. The value can be either an integer or a string:

number | string |
---|---|

0 | 'blank' |

1 | 'solid' |

2 | 'dashed' |

3 | 'dotted' |

4 | 'dotdash' |

5 | 'longdash' |

6 | 'twodash' |

## additional point set

## stacked area chart

## overlapping area chart

## 3d scatter plot

## bubble chart

## scatter plot matrix

## contour plot

# MATLAB

Octave Manual

MATLAB Documentation

Differences between Octave and MATLAB

Octave-Forge Packages

The basic data type of MATLAB is a matrix of floats. There is no distinction between a scalar and a 1x1 matrix, and functions that work on scalars typically work on matrices as well by performing the scalar function on each entry in the matrix and returning the results in a matrix with the same dimensions. Operators such as the logical operators ('&' '|' '!'), relational operators ('==', '!=', '<', '>'), and arithmetic operators ('+', '-') all work this way. However the multiplication '*' and division '/' operators perform matrix multiplication and matrix division, respectively. The `.*` and `./` operators are available if entry-wise multiplication or division is desired.

Floats are by default double precision; single precision can be specified with the *single* constructor. MATLAB has convenient matrix literal notation: commas or spaces can be used to separate row entries, and semicolons or newlines can be used to separate rows.

Arrays and vectors are implemented as single-row (1xn) matrices. As a result an *n*-element vector must be transposed before it can be multiplied on the right of a *mxn* matrix.

Numeric literals that lack a decimal point such as *17* and *-34* create floats, in contrast to most other programming languages. To create an integer, an integer constructor which specifies the size such as *int8* and *uint16* must be used. Matrices of integers are supported, but the entries in a given matrix must all have the same numeric type.

Strings are implemented as single-row (1xn) matrices of characters, and as a result matrices cannot contain strings. If a string is put in matrix literal, each character in the string becomes an entry in the resulting matrix. This is consistent with how matrices are treated if they are nested inside another matrix. The following literals all yield the same string or 1xn matrix of characters:

```
'foo'
[ 'f' 'o' 'o' ]
[ 'foo' ]
[ [ 'f' 'o' 'o' ] ]
```

*true* and *false* are functions which return matrices of ones and zeros. The ones and zeros have type *logical* instead of *double*, which is created by the literals 1 and 0. Other than having a different class, the 0 and 1 of type *logical* behave the same as the 0 and 1 of type *double*.

MATLAB has a tuple type (in MATLAB terminology, a cell array) which can be used to hold multiple strings. It can also hold values with different types.

# R

An Introduction to R

Advanced R Programming

The Comprehensive R Archive Network

The basic data types of R are vectors of floats, vectors of strings, and vectors of booleans. There is no distinction between a scalar and a vector with one entry in it. Functions and operators which accept a scalar argument will typically accept a vector argument, returning a vector of the same size with the scalar operation performed on each the entries of the original vector.

The scalars in a vector must all be of the same type, but R also provides a *list* data type which can be used as a tuple (entries accessed by index), record (entries accessed by name), or even as a dictionary.

In addition R provides a *data frame* type which is a list (in R terminology) of vectors all of the same length. Data frames are equivalent to the data sets of other statistical analysis packages.

# NumPy

NumPy and SciPy Documentation

matplotlib intro

NumPy for Matlab Users

Pandas Documentation

Pandas Method/Attribute Index

NumPy is a Python library which provides a data type called `array`. It differs from the Python `list` data type in the following ways:

- N-dimensional. Although the
`list`type can be nested to hold higher dimension data, the`array`can hold higher dimension data in a space efficient manner without using indirection. - homogeneous. The elements of an
`array`are restricted to be of a specified type. The NumPy library introduces new primitive types not available in vanilla Python. However, the element type of an array can be`object`which permits storing anything in the array.

In the reference sheet the array section covers the vanilla Python `list` and the multidimensional array section covers the NumPy `array`.

*List the NumPy primitive types*

SciPy, Matplotlib, and Pandas are libraries which depend on Numpy.