a side-by-side reference sheet

sheet one: grammar and invocation | variables and expressions | arithmetic and logic | strings | regexes | dates and time
                 arrays | dictionaries | functions | execution control

sheet two: file handles | files | directories | processes and environment | libraries and namespaces | objects | reflection
                 net and web | unit tests | debugging and profiling | java interop | contact

File Handles

standard file handles

The names for standard input, standard output, and standard error.

read line from stdin

How to read a line from standard input.

The illustrated function read the standard input stream until a end-of-line marker is found or the end of the stream is encountered. Only in the former case will the returned string be terminated by an end-of-line marker.

php:

fgets takes an optional second parameter to specify the maximum line length. If the length limit is encountered before a newline, the string returned will not be newline terminated.

ruby:

gets takes an optional parameter to specify the maximum line length. If the length limit is encountered before a newline, the string returned will not be newline terminated.

end-of-file behavior

What happens when attempting to read a line and the seek point is after the last newline or at the end of the file.

chomp

Remove a newline, carriage return, or carriage return newline pair from the end of a line if there is one.

php:

chop removes all trailing whitespace. It is an alias for rtrim.

perl:

chomp modifies its argument, which thus must be a scalar, not a string literal.

python:

Python strings are immutable. rstrip returns a modified copy of the string. rstrip('\r\n') is not identical to chomp because it removes all contiguous carriage returns and newlines at the end of the string.

ruby:

chomp! modifies the string in place. chomp returns a modified copy.

write line to stdout

How to write a line to standard out. The line will be terminated by an operating system appropriate end of line marker.

python:

print appends a newline to the output. To suppress this behavior, put a trailing comma after the last argument. If given multiple arguments, print joins them with spaces.

In Python 2 print parses as a keyword and parentheses are not required:

print "Hello, World!"

ruby:

puts appends a newline to the output. print does not.

write formatted string to stdout

How to format variables and write them to standard out.

The function printf from the C standard library is a familiar example. It has a notation for format strings which uses percent signs %. Many other languages provide an implementation of printf.

open file for reading

How to open a file for reading.

perl:

Before Perl 5.6 one would create and store file handles in bare words:

open FIN, "/etc/hosts";
open FOUT, ">/tmp/test";

my $line = <FIN>;
print FOUT $line;

close FIN;
close FOUT;

To pass a bare word file handle to a function one would use a combination of the typeglob operator and the reference operator to store the file handle in a scalar:

sub log_msg {
  my $fh = shift;
  my $msg = shift;

  print $fh $msg;
}

open(LOG, ">>/tmp/test.log");

log_msg(\*LOG, "log opened for append\n");

In Perl 5.6 and later, the open function, when given an undefined scalar (or a scalar declaration) as a first argument will create and store the file handle in that scalar.

Using scalars to manipulate file handles is preferred.

• Scalars can be passed to functions.
• Scalars can have local scope whereas bare words are always global.
• A file handle stored in a scalar is implicitly closed when the scalar goes out of scope.
• Under 'use strict', misspelled scalar names are caught by the interpreter, but misspelled bare words are not.

ruby:

When File.open is given a block, the file is closed when the block terminates.

open file for writing

How to open a file for writing. If the file exists its contents will be overwritten.

open file for appending

How to open a file with the seek point at the end of the file. If the file exists its contents will be preserved.

close file

How to close a file.

close file implicitly

How to have a file closed when a block is exited.

python:

File handles are closed when the variable holding them is garbage collected, but there is no guarantee when or if a variable will be garbage collected.

ruby:

File handles are closed when the variable holding them is garbage collected, but there is no guarantee when or if a variable will be garbage collected.

i/o errors

How I/O errors are treated.

read line

How to read up to the next newline in a file.

iterate over file by line

How to iterate over a file line by line.

read file into array of string

How to put the lines of a file into an array of strings.

read file into string

How to put the contents of a file into a single string.

write string

How to write a string to a file handle.

write line

How to write a line to a file handle. An operating system appropriate end-of-line marker is appended to the output.

**php:

Newlines in strings are translated to the operating system appropriate line terminator unless the file handle was opened with a mode string that contained 'b'.

perl:

Newlines in strings are translated to the operating system appropriate line terminator unless the file handle has been set to binary mode with the binmode function.

python:

When file handles are opened with the mode strings 'r', 'w', or 'a', the file handle is in text mode. In text mode the operating system line terminator is translated to '\n' when reading and '\n' is translated back to the operating system line terminator when writing. The standard file handles sys.stdin, sys.stdout, and sys.stderr are opened in text mode.

When file handles are opened with the mode strings 'rb', 'rw', or 'ra', the file handle is in binary mode and line terminator translation is not performed. The operating system line terminator is available in os.linesep.

flush file handle

How to flush a file handle that has been written to.

end-of-file test

How to test whether the seek point of a file handle is at the end of the file.

get and set file handle position

How to get or set the file handle seek point.

The seek point is where the next read on the file handle will begin. The seek point is measured in bytes starting from zero.

open temporary file

How to get a file handle to a file that will be removed automatically sometime between when the file handle is closed and the interpreter exits.

The file is guaranteed not to have existed before it was opened.

The file handle is opened for both reading and writing so that the information written to the file can be recovered
by seeking to the beginning of the file and reading from the file handle.

On POSIX operating systems it is possible to unlink a file after opening it. The file is removed from the directory but continues to exist as long as the
file handle is open. This guarantees that no other process will be able to read or modify the file contents.

php:

Here is how to create a temporary file with a name:

$path = tempnam(sys_get_temp_dir(), "");
$f = fopen($path, "w+");

perl:

How to unlink a temporary file on open:

use File::Temp;

$f = File::Temp->new(UNLINK=>1);

python:

To unlink a temporary file on open, used TemporaryFile instead of NamedTemporaryFile:

import tempfile

f = tempfile.TemporaryFile()

in memory file

How to create a file descriptor which writes to an in-memory buffer.

python:

StringIO also supports the standard methods for reading input. To use them the client must first seek to the beginning of the in-memory file:

f = StringIO()
f.write('lorem ipsum\n')
f.seek(0)
r.read()

Files

file test, regular file test

How to test whether a file exists; how to test whether a file is a regular file (i.e. not a directory, special device, or named pipe).

file size

How to get the file size in bytes.

is file readable, writable, executable

How to test whether a file is readable, writable, or executable.

python:

The flags can be or'ed to test for multiple permissions:

os.access('/etc/hosts', os.R_OK | os.W_OK | os.X_OK)

set file permissions

How to set the permissions on the file.

For Perl, Python, and Ruby, the mode argument is in the same format as the one used with the Unix chmod command. It uses bitmasking to get the various permissions which is why it is normally an octal literal.

The mode argument should not be provided as a string such as "0755". Python and Ruby will raise an exception if a string is provided. Perl will convert "0755" to 755 and not 0755 which is equal to 493 in decimal.

copy file, remove file, rename file

How to copy a file; how to remove a file; how to rename a file.

create symlink, symlink test, readlink

How to create a symlink; how to test whether a file is a symlink; how to get the target of a symlink.

generate unused file name

How to generate an unused file name. The file is created to avoid a race condition with another process looking for an unused file name.

The file is not implicitly deleted.

Directories

working directory

How to get and set the working directory.

build pathname

How to construct a pathname without hard coding the system file separator.

dirname and basename

How to extract the directory portion of a pathname; how to extract the non-directory portion of a pathname.

absolute pathname

How to get the get the absolute pathname for a pathname. If the pathname is relative the working directory will be appended.

In the examples provided, if /foo/bar is the working directory and .. is the relative path, then the return value is foo

perl:

File::Spec->rel2abs is similar to Cwd::abs_path except that paths with ".." will not be simplified. If the working directory is /foo/bar then the output of File::Spec->rel2abs("..") is

/foo/bar/..

iterate over directory by file

How to iterate through the files in a directory.

In PHP, Perl, and Ruby, the files representing the directory itself . and the parent directory .. are returned.

php:

The code in the example will stop if a filename which evaluates as FALSE is encountered. One such filename is "0". A safer way to iterate through the directory is:

if ($dir = opendir("/etc")) {
  while (FALSE !== ($file = readdir($dir))) {
    echo "$file\n";
  }
  closedir($dir);
}

python:

file() is the file handle constructor. file can be used as a local variable name but doing so hides the constructor. It can still be invoked by the synonym open(), however.

os.listdir() does not return the special files . and .. which represent the directory itself and the parent directory.

glob paths

How to iterate over files using a glob pattern.

Glob patterns employ these special characters:

Use glob patterns instead of simple directory iteration when

• dot files, including the directory itself (.) and the parent directory (..), should skipped
• a subset of the files in a directory, where the subset can be specified with a glob pattern, is desired
• files from multiple directories, where the directories can be specified with a glob pattern, are desired
• the full pathnames of the files is desired

php:

glob takes a second argument for flags. The flag GLOB_BRACE enables brace notation.

python:

glob.glob returns a list. glob.iglob accepts the same arguments and returns an iterator.

ruby:

Ruby globs support brace notation.

A brace expression matches any of the comma separated strings inside the braces.

Dir.glob("/{bin,etc,usr}/*").each do |path|
  puts path
end

make directory

How to create a directory.

If needed, the examples will create more than one directory.

No error will result if a directory at the pathname already exists. An exception will be raised if the pathname is occupied by a regular file, however.

recursive copy

How to perform a recursive copy. If the source is a directory, then the directory and all its contents will be copied.

remove empty directory

How to remove an empty directory. The operation will fail if the directory is not empty.

remove directory and contents

How to remove a directory and all its contents.

directory test

How to determine if a pathname is a directory.

generate unused directory

How to generate an unused directory. The directory is created to avoid a race condition with another process looking for an unused directory.

The directory is not implicitly deleted.

ruby:

When Dir.mktmpdir is provided with a block the directory is deleted after the block finishes executing:

require 'tmpdir'
require 'fileutils'

Dir.mktmpdir("/tmp/foo") do |path|
  puts path
  FileUtils.cp("/etc/hosts", "#{path}/hosts")
end

system temporary file directory

The name of the system provided directory for temporary files.

On Linux the directory is often /tmp, and the operating system is often configured to delete the contents of /tmp at boot.

Processes and Environment

command line arguments

How to access arguments provided at the command line when the script was run; how to get the name of the script.

command line options

How to process command line options.

We describe the style used by getopt_long from the C standard library. The characteristics of this style are:

• Options can be short or long. Short options are a single character preceded by a hyphen. Long options are a word preceded by two hyphens.
• A double hyphen by itself can be used to terminate option processing. Arguments after the double hyphen are treated as positional arguments and can start with a hyphen.
• Options can be declared to be with or without argument. Options without argument are used to set a boolean value to true.
• Short options without argument can share a hyphen.
• Long options can be separated from their argument by a space or an equals sign (=). Short options can be separated from their argument by nothing, a space, or an equals sign (=).

The option processing function should identify the positional arguments. These are the command line arguments which are not options, option arguments, or the double hyphen used to terminate option processing. getopt_long permits options to occur after positional arguments.

perl:

The type of an option argument can be specified as a string, integer, or float. If the argument cannot be converted to an integer or float a warning is written to standard error and GetOptions() returns a non-zero value.

if (GetOptions("file=s" => \$file,
               "count=i" => \$count,
               "ratio=f" => \$ratio)) {
  ...

python:

The type of an argument can be specified using the named parameter type:

parser.add_argument('--count', '-c', dest='count', type=int)
parser.add_argument('--ratio', '-r', dest='ratio', type=float)

If the argument cannot be converted to the type, the script prints out a usage statement and exits with a non-zero value.

The default value is None, but this can be changed using the named parameter default:

parser.add_argument('--file', '-f', dest='file', default='tmpfile')
parser.add_argument('--count', '-c', dest='count', type=int, default=1)
parser.add_argument('--ratio', '-r', dest='ratio', type=float, default=0.5)

get and set environment variable

How to get and set an environment variable. If an environment variable is set the new value is inherited by child processes.

php:

putenv returns a boolean indicating success. The command can fail because when PHP is running in safe mode only some environment variables are writable.

get pid, parent pid

How to get the process id of the interpreter process; how to get the id of the parent process.

perl:

When use English is in effect, the interpreter pid is in the variables $PID and $PROCESS_ID.

ruby:

The process pid is also available in the global variable $$.

get user id and name

How to get the user id of the interpreter process; how to get the username associated with the user id.

When writing a setuid application on Unix, there is a distinction between the real user id and the effective user id. The code examples return the real user id.

The process may be able to determine the username by inspecting environment variables. A POSIX system is required to set the environment variable LOGNAME at login. Unix systems often set USER at login, and Windows systems set %USERNAME%. There is nothing to prevent the user from altering any of these environment variables after login. The methods illustrated in the examples are thus more secure.

perl:

When use English is in effect the user id is also available in the variables $UID and $REAL_USER_ID.

The effective user id is available in $>. When use English is in effect the effective user id is also available in $EUID and $EFFECTIVE_USER_ID.

python:

How to get the effective user id:

os.geteuid()

ruby:

How to get the effective user id:

Process.euid

exit

python:

It is possible to register code to be executed upon exit:

import atexit
atexit.register(print, "goodbye")

It is possible to terminate a script without executing registered exit code by calling os._exit.

ruby:

It is possible to register code to be executed upon exit:

at_exit { puts "goodbye" }

The script can be terminated without executing registered exit code by calling exit!.

set signal handler

How to register a signal handling function.

executable test

How to test whether a file is executable.

external command

How to execute an external command.

escaped external command

How to prevent shell injection.

backticks

How to invoke an external command and read its output into a variable.

The use of backticks for this operation goes back to the Bourne shell (1977).

perl:

The qx operator can be used with any delimiter. If the opening delimiter is (, [, or {, the closing delimiter must be ), ], or }.

python:

A more concise solution is:

file = os.popen('ls -l /tmp').read()

os.popen was marked as deprecated in Python 2.6 but it is still available in Python 2.7 and Python 3.2.

ruby:

%x can be used with any delimiter. If the opening delimiter is (, [, or {, the closing delimiter must be ), ], or }.

Libraries and Namespaces

Terminology used in this sheet:

• library: code in its own file that can be included, loaded, or linked by client code.
• client: code which calls code in a separate file.
• top-level file or top-level script: the file containing the code in the program which executes first.
• load: to add definitions in a file to the text of a running process.
• namespace: a set of names that can be imported as a unit.
• import: to add definitions defined elsewhere to a scope.
• unqualified import: to add definitions to a scope using the same identifiers as where they are defined.
• qualified import: to add definitions to a scope. The identifiers in the scope are derived from the original identifiers in a formulaic manner. Usually the name of the namespace is added as a prefix.
• label: one of the parts of a qualified identifier.
• alias import: to add a definition to a scope under an identifier which is specified in the import statement.
• package: one or more libraries that can be installed by a package manager.

load library

Execute the specified file. Normally this is used on a file which only contains declarations at the top level.

perl:

The last expression in a perl library must evaluate to true.

php:

include_once behaves like require_once except that it is not fatal if an error is encountered executing the library.

load library in subdirectory

How to load a library in a subdirectory of the library path.

hot patch

How to reload a library. Altered definitions in the library will replace previous versions of the definition.

php:

Also include.

load error

How errors which are encountered while loading libraries are handled.

main routine in library

How to put code in a library which executes only when the file is run as a top-level script.

library path

The library path is a list of directory paths which are searched when loading libraries.

library path environment variable

How to augment the library path by setting an environment variable before invoking the interpreter.

library path command line option

How to augment the library path by providing a command line option when invoking the interpreter.

simple global identifiers

multiple label identifiers

label separator

The punctuation used to separate the labels in the full name of a subnamespace.

root namespace definition

namespace declaration

How to declare a section of code as belonging to a namespace.

subnamespace declaration

How to declare a section of code as belonging to a subnamespace.

import namespace

import subnamespace

import all definitions in namespace

How to import all the definitions in a namespace.

import definitions

How to import specific definitions from a namespace.

list installed packages, install a package

How to show the installed 3rd party packages, and how to install a new 3rd party package.

perl

cpanm is an alternative to cpan which is said to be easier to use.

How to use cpan to install cpanm:

$ sudo cpan -i App::cpanminus

How to install a module with cpanm:

$ sudo cpanm Moose

python

Two ways to list the installed modules and the modules in the standard library:

$ pydoc modules

$ python
>>> help('modules')

Most 3rd party Python code is packaged using distutils, which is in the Python standard library. The code is placed in a directory with a setup.py file. The code is installed by running the Python interpreter on setup.py:

package specification format

The format of the file used to specify a package.

python:

distutils.core reference

How to create a Python package using distutils. Suppose that the file foo.py contains the following code:

def add(x, y):
    return x+y

In the same directory as foo.py create setup.py with the following contents:

#!/usr/bin/env python

from distutils.core import setup

setup(name='foo',
      version='1.0',
      py_modules=['foo'],
     )

Create a tarball of the directory for distribution:

$ tar cf foo-1.0.tar foo
$ gzip foo-1.0.tar

To install a tar, perform the following:

$ tar xf foo-1.0.tar.gz
$ cd foo
$ sudo python setup.py install

If you want people to be able to install the package with pip, upload the tarball to the Python Package Index.

ruby:

gemspec attributes

For an example of how to create a gem, create a directory called foo. Inside it create a file called lib/foo.rb which contains:

def add(x, y)
  x + y
end

Then create a file called foo.gemspec containing:

spec = Gem::Specification.new do |s|
  s.name = 'foo'
  s.authors = 'Joe Foo'
  s.version = '1.0'
  s.summary = 'a gem'
  s.files = Dir['lib/*.rb']
end

To create the gem, run this command:

$ gem build foo.gemspec

A file called foo-1.0.gem is created. To install foo.rb run this command:

$ gem install foo-1.0.gem

Objects

define class

php:

Properties (i.e. instance variables) must be declared public, protected, or private. Methods can optionally be declared public, protected, or private. Methods without a visibility modifier are public.

perl:

The sheet shows how to create objects using the CPAN module Moose. To the client of an object, Moose objects and traditional Perl objects are largely indistinguishable. Moose provides convenience functions to aid in the definition of a class, and as a result a Moose class definition and a traditional Perl class definition look quite different.

The most common keywords used when defining a Moose class are has, extends, subtype.

The before, after, and around keywords are used to define method modifiers. The with keyword indicates that a Moose class implements a role.

The no Moose; statement at the end of a Moose class definition removes class definition keywords, which would otherwise be visible to the client as methods.

Here is how to define a class in the traditional Perl way:

package Int;

sub new {
  my $class = shift;
  my $v = $_[0] || 0;
  my $self = {value => $v};
  bless $self, $class;
  $self;
}

sub value {
  my $self = shift;
  if ( @_ > 0 ) {
    $self->{'value'} = shift;
  }
  $self->{'value'};
}

sub add {
  my $self = shift;
  $self->value + $_[0];
}

sub DESTROY {
  my $self = shift;
  my $v = $self->value;
  print "bye, $v\n";
}

python:

As of Python 2.2, classes are of two types: new-style classes and old-style classes. The class type is determined by the type of class(es) the class inherits from. If no superclasses are specified, then the class is old-style. As of Python 3.0, all classes are new-style.

New-style classes have these features which old-style classes don't:

• universal base class called object.
• descriptors and properties. Also the __getattribute__ method for intercepting all attribute access.
• change in how the diamond problem is handled. If a class inherits from multiple parents which in turn inherit from a common grandparent, then when checking for an attribute or method, all parents will be checked before the grandparent.

create object

How to create an object.

get and set attribute

How to get and set an attribute.

perl:

Other getters:

$i->value()
$i->{'value'}

Other setters:

$i->{'value'} = $v;

python:

Defining explicit setters and getters in Python is considered poor style. If it becomes necessary to extra logic to attribute, this can be achieved without disrupting the clients of the class by creating a property:

def getValue(self):
  print("getValue called")
  return self.__dict__['value']
def setValue(self,v):
  print("setValue called")
  self.__dict__['value'] = v
value = property(fget=getValue, fset = setValue)

instance variable accessibility

How instance variable access works.

define method

How to define a method.

invoke method

How to invoke a method.

perl:

If the method does not take any arguments, the parens are not necessary to invoke the method.

destructor

How to define a destructor.

perl:

Perl destructors are called when the garbage collector reclaims the memory for an object, not when all references to the object go out of scope. In traditional Perl OO, the destructor is named DESTROY, but in Moose OO it is named DEMOLISH.

python:

A Python destructor is not guaranteed to be called when all references to an object go out of scope, but apparently this is how the CPython implementations work.

ruby:

Ruby lacks a destructor. It is possible to register a block to be executed before the memory for an object is released by the garbage collector. A ruby
interpreter may exit without releasing memory for objects that have gone out of scope and in this case the finalizer will not get called. Furthermore, if the finalizer block holds on to a reference to the object, it will prevent the garbage collector from freeing the object.

method missing

How to handle when a caller invokes an undefined method.

php:

Define the method __callStatic to handle calls to undefined class methods.

python:

__getattr__ is invoked when an attribute (instance variable or method) is missing. By contrast, __getattribute__, which is only available in Python 3, is always invoked, and can be used to intercept access to attributes that exist. __setattr__ and __delattr__ are invoked when attempting to set or delete attributes that don't exist. The del statement is used to delete an attribute.

ruby:

Define the method self.method_missing to handle calls to undefined class methods.

inheritance

How to use inheritance.

perl:

Here is how inheritance is handled in traditional Perl OO:

package Counter;

our @ISA = "Int";

my $instances = 0;
our $AUTOLOAD;

sub new {
  my $class = shift;
  my $self = Int->new(@_);
  $instances += 1;
  bless $self, $class;
  $self;
}

sub incr {
  my $self = shift;
  $self->value($self->value + 1);
}
sub instances {
  $instances;
}

sub AUTOLOAD {
  my $self = shift;
  my $argc = scalar(@_);
  print "undefined: $AUTOLOAD " .
    "arity: $argc\n";
}

define class method

invoke class method

How to invoke a class method.

operator overloading

How to define the behavior of the binary operators.

method alias

How to create an alias for a method.

ruby:

Ruby provides the keyword alias and the method alias_method in the class Module. Inside a class body they behave identifically. When called from inside a method alias has no effect but alias_method works as expected. Hence some recommend always using alias_method.

Reflection

object id

How to get an identifier for an object or a value.

inspect type

php:

The PHP manual says that the strings returned by gettype are subject to change and advises using the following predicates instead:

is_null
is_bool
is_numeric
is_int
is_float
is_string
is_array
is_object
is_resource

perl:

ref returns the empty string when the argument is not a scalar containing a reference. If the argument is a reference, ref returns the package name if it points to a blessed object. Otherwise it returns the name of the built-in type.

basic types

php:

All possible return values of gettype are listed.

perl:

All the built-in types are listed.

inspect class

How to get the class of an object.

inspect class hierarchy

has method?

perl:

$a->can() returns a reference to the method if it exists, otherwise it returns undef.

python:

hasattr(o,'reverse') will return True if there is an instance variable named 'reverse'.

message passing

eval

How to interpret a string as code and return its value.

php:

The value of the string is the value of of the return statement that terminates execution. If execution falls off the end of the string without encountering a return statement, the eval evaluates as NULL.

python:

The argument of eval must be an expression or a SyntaxError is raised. The Python version of the mini-REPL is thus considerably less powerful than the versions for the other languages. It cannot define a function or even create a variable via assignment.

list object methods

list object attributes

perl:

keys %$a assumes the blessed object is a hash reference.

python:

dir(o) returns methods and instance variables.

pretty print

How to display the contents of a data structure for debugging purposes.

source line number and file name

How to get the current line number and file name of the source code.

command line documentation

How to get documentation from the command line.

ruby:

Searching for Math.atan2 will return either class method or instance method documentation. If there is documentation for both one can be specific with the following notation:

$ ri Math::atan2
$ ri Math#atan2

Net and Web

get local hostname, dns lookup, reverse dns lookup

How to get the hostname and the ip address of the local machine without connecting to a socket.

The operating system should provide a method for determining the hostname. Linux provides the uname system call.

A DNS lookup can be performed to determine the IP address for the local machine. This may fail if the DNS server is unaware of the local machine or if the DNS server has incorrect information about the local host.

A reverse DNS lookup can be performed to find the hostname associated with an IP address. This may fail for the same reasons a forward DNS lookup might fail.

http get

How to make an HTTP GET request and read the response into a string.

serve working directory

A command line invocation to start a single process web server which serves the working directory at http://localhost:8000.

perl:

The following webserver serves files but does not provide directory listings:

$ sudo cpan -i IO::All

$ perl -MIO::All -e 'io(":8000")->fork->accept->(sub { $_[0] < io(-x $1 ? "./$1 |" : $1) if /^GET \/(.*) / })'

absolute url

How to construct an absolute URL from a base URL and a relative URL as documented in RFC 1808.

When constructing the absolute URL, the rightmost path component of the base URL is removed unless it ends with a slash /. The query string and fragment of the base URL are always removed.

If the relative URL starts with a slash / then the entire path of the base URL is removed.

If the relative URL starts with one or more occurrences of ../ then one or more path components are removed from the base URL.

The base URL and the relative URL will be joined by a single slash / in the absolute URL.

php:

Here is a PHP function which computes absolute urls.

parse url

How to extract the protocol, host, port, path, query string, and fragment from a URL. How to extract the parameters from the query string.

python:

urlparse can also be used to parse FTP URLs:

up = urlparse.urlparse('ftp://foo:bar@google.com/baz;type=binary')

# 'foo'
up.username

# 'bar'
up.password

# 'type=binary'
up.params

ruby:

How to parse an FTP URL:

up = URI('ftp://foo:bar@google.com/baz;type=binary')

# "foo"
 up.user

# up.password
"bar"

# "binary"
up.typecode

url encode/decode

How to URL encode and URL unencode a string.

URL encoding, also called percent encoding, is described in RFC 3986. It replaces all characters except for the letters, digits, and a few punctuation marks with a percent sign followed by their two digit hex encoding. The characters which are not escaped are:

A-Z a-z 0-9 - _ . ~

URL encoding can be used to encode UTF-8, in which case each byte of a UTF-8 character is encoded separately.

When form data is sent from a browser to a server via an HTTP GET or an HTTP POST, the data is percent encoded but spaces are replaced by plus signs + instead of %20. The MIME type for form data is application/x-www-form-urlencoded.

perl:

CGI::escape replaces spaces with %20. CGI::unescape will replace both + and %20 with a space, however.

python:

In Python 3 the functions quote_plus, unquote_plus, quote, and unquote moved from urllib to urllib.parse.

urllib.quote replaces a space character with %20.

urllib.unquote does not replace + with a space character.

base64 encode/decode

How to encode binary data in ASCII using the Base64 encoding scheme.

A popular Base64 encoding is the one defined by RFC 2045 for MIME. Every 3 bytes of input is mapped to 4 of these characters: [A-Za-z0-9/+].
If the input does not consist of a multiple of three characters, then the output is padded with one or two hyphens: =.

Whitespace can inserted freely into Base64 output; this is necessary to support transmission by email. When converting Base64 back to binary whitespace is ignored.

json generate/parse

How to encode JSON sdata in a JSON string; how to decode such a string.

JSON data consists of objects, arrays, and JSON values. Objects are dictionaries in which the keys are strings and the values are JSON values. Arrays contain JSON values. JSON values can be objects, arrays, strings, numbers, true, false, or null.

A JSON string is JSON data encoded using the corresponding literal notation used by JavaScript source code.

JSON strings are sequences of Unicode characters. The following backslash escape sequences are supported:

  \" \\ \/ \b \f \n \r \t \uhhhh.

generate xml

How to build an XML document.

An XML document can be constructed by concatenating strings, but the techniques illustrated here guarantee the result to be well-formed XML.

parse xml

How to parse XML and extract nodes using XPath.

ruby:

Another way of handling an XPath expression which matches multiple nodes:

XPath.each(doc,"/a/b/c") do |node|
  puts node.text
end

parse html

How to parse an HTML document.

Unit Tests

test class

How to define a test class and make a truth assertion.

The argument of a truth assertion is typically an expression. It is a good practice to include a failure message as a second argument which prints out variables in the expression.

perl:

If there is more than one assertion in a test, then set the Test attribute appropriately in the test method signature to quiesce a warning:

sub test_01 : Test(2) {
  ok(1);
  ok(2);
}

run tests; run test method

How to run all the tests in a test class; how to run a single test from the test class.

equality assertion

How to test for equality.

python:

Note that assertEquals does not print the values of its first two arguments when the assertion fails. A third argument can be used to provide a more informative failure message.

approximate assertion

How to assert that two floating point numbers are approximately equal.

regex assertion

How to test that a string matches a regex.

exception assertion

How to test whether an exception is raised.

mock method

How to create a mock method.

A mock method is used when calling the real method from a unit test would be undesirable. The method that is mocked is not in the code that is being tested, but rather a library which is used by that code. Mock methods can raise exceptions if the test fails to invoke them or if the wrong arguments are provided.

python:

assert_called_once_with can takes the same number of arguments as the method being mocked.

If the mock method was called multiple times, the method assert_called_with can be used in place of asert_called_once_with to make an assertion about the arguments that were used in the most recent call.

A mock method which raises an exception:

foo = Foo()
foo.run = mock.Mock(side_effect=KeyError('foo'))

with self.assertRaises(KeyError):
  foo.run(13)

foo.run.assert_called_with(13)

ruby:

The with method takes the same number of arguments as the method being mocked.

Other methods are available for use in the chain which defines the assertion. The once method can be replaced by never or twice. If there is uncertainty about how often the method will be called one can used at_least_once, at_least(m), at_most_once, at_most(n) to set lower or upper bounds. times(m..n) takes a range to set both the lower and upper bound.

A mock method which raises an exception:

    foo = mock()
    foo.expects(:run).
      raises(exception = RuntimeError, message = 'bam!').
      with(13).
      once

    assert_raises(RuntimeError) do
      foo.run(13)
    end

There is also a method called yields which can be used in the chain which defines the assertion. It makes the mock method yield to a block. It takes as arguments the arguments it passes to the block.

setup

How to define a setup method which gets called before every test.

teardown

How to define a cleanup method which gets called after every test.

Debugging and Profiling

check syntax

How to check the syntax of code without executing it.

flags for stronger and strongest warnings

Flags to increase the warnings issued by the interpreter.

perl:

The

use warnings;

pragma is the same as the -w flag except that warnings are only issued for constructs in the current scope.

python:

The -t flag warns about inconsistent use of tabs in the source code. The -3 flag is a Python 2.X option which warns about syntax which is no longer valid in Python 3.X.

lint

A lint tool.

source cleanup

A tool which detects or removes semantically insignificant variation in the source code.

run debugger

How to run a script under the debugger.

debugger commands

A selection of commands available when running the debugger. The gdb commands are provided for comparison.

benchmark code

How to run a snippet of code repeatedly and get the user, system, and total wall clock time.

profile code

How to run the interpreter on a script and get the number of calls and total execution time for each function or method.

perl:

perl -d:DProf writes the profiling information to the file tmon.out in the current directory. dprofpp reads that file.

Java Interoperation

Both Python and Ruby have JVM implementations. It is possible to compile both Python code and Ruby code to Java bytecode and run it on the JVM. It is also possible to run a version of the Python interpreter or the Ruby interpreter on the JVM which reads Python code or Ruby code, respectively.

version

Version of the scripting language JVM implementation used in this reference sheet.

repl

Command line name of the repl.

interpreter

Command line name of the interpreter.

compiler

Command line name of the tool which compiles source to java byte code.

prologue

Code necessary to make java code accessible.

new

How to create a java object.

method

How to invoke a java method.

import

How to import names into the current namespace.

import non-bundled java library

How to import a non-bundled Java library

shadowing avoidance

How to import Java names which are the same as native names.

convert native array to java array

How to convert a native array to a Java array.

are java classes subclassable?

Can a Java class be subclassed?

are java classes open?

Can a Java array be monkey patched?