Introduction

Standard Libraries
asutil
core
dump
dynamictypes
elf
nsgraph
nsutil
Regular Expression Library
Red Black Tree
snames
gdb

Cinquecento Libraries

Introduction

This document describes various libraries provided with the L1 implementation of Cinquecento. It is not complete. See the L1 Manual for more details on Cinquecento and L1.

Standard Libraries

The L1 implementation of Cinquecento comes with several standard libraries stored in the lib/ subdirectory. Since they are included in l1's load path, these libraries are loaded via the form:

@include <library>

This section gives non-exhaustive describions for several of the provided libraries.

asutil.cqct

asutil.cqct provides code for loading, saving, and copying address spaces.

asload(filename)
filename: string
address space
Loads the address space saved to the file in filename.
asload_str(str)
str: string
address space
Loads the address space saved in the string str.
assave(as, filename)
as: address space filename: string
undefined
Saves the address space as to the file filename.
assave_str(as)
as: address space
string
Saves the address space as to a string, and returns that string.
ascopy(as)
as: address space
address space
Creates a copy of the address space as, and returns the copy. No changes to the returned address space affect the passed address space.
mkasoff(as, offset)
as: address space offset: int
address space
Creates a new address space with the same contents as the the provided address space but with addresses changed by offset. That is, address 0 of the new adddress space will be offset offset of as.
core.cqct

core.cqct provides functionality for loading a core file as an address space.

mkcoreas(fname)
fname: string
address space
Returns an domain for the provided core file representing that program at the point at which the core file was created. The domain defines the functions getXXX, where XXX can be any of the standard register names.
dump.cqct

dump.cqct provides code for printing out namespaces and the values associated with typed pointers.

printtype(t)
t: ctype
string
Returns a string representation of the provided ctype that can be included inside a @names directive.
dumpns(dn)
dn: Domain or name space to be printed.
nil
Prints all types in the given domain or name space to standard out.
objectStr(x)
x: Pointer to a cvalue.
string
Pretty-prints the object pointed to by x based on x's ctype to a string, and returns that string.
dynamictypes.cqct

This file provides the ability to create dynamic types, or types that change based on their content. For example, consider the following vector type:

struct Vector {
        unsigned int size;
        int data[];
};
The data pointer is actually an array of length size. With dynamic typing, one can specify types such that a pointer p pointing to an object of type Vector can be translated into the appropriate type using the refine function. In this case, that type will be:
stuct Vector {
        unsigned int size;
	int data[p->size];
};
The basic idea is for the person defining the type to include an extra mapping in fields for the keyword "refine", which specifies a function for performing the transformation. Then for a pointer to an object p, one can call refine(p) to create a pointer that points to the refined version of the type of p. For a complete example, see the example function in the file dynamictypes.cqct.
refine(p)
p: A cvalue pointer.
A cvalue pointer.
Returns a pointer to the same address as is pointed to by p, but points to a refined version of p's type. The newly refined type inherits any refinement properties from the old type so that further calls to refine will behave as expected. If any part of the object p points to is unmapped, or if any part of the refined structure p would have pointed to on return is unmapped, then no changes are made.
elf.cqct

elf.cqct contains code for handling elf files. Much of this function provides machinery to keep the client code unaware of whether or not the elf file underconsideration contains a 32 or 64 bit address space. The recode bd type, which is instantiated through mkelfrec, is where the elf domain is contained. The elfsym types, which are created via getcqctelfsym contain information about specific elf symbols.

Record bd created by mkelfrec
rootns: namespace for the elf file
elf: domain for the elf file
Vector elfsym is used to store information on elf symbols. These are created by getcqctelfsym, and can be printed with showsym. That information can be harvested directly by the following functions:
elfname(esym): returns the name (field 0)
elfaddr(esym): returns the address (field 1)
elftype(esym): returns the type (field 2)
elfvisibility(esym): returns the visibility (field 3)
elfsize(esym): returns the size (field 4)
elfinfo(esym): returns the info (field 5)
elfother(esym): returns the other stuff (field 6)
elfshndx(esym): returns the type (field 7)
dumpsym(name, esym)
name: string
esym: struct ElfXX_Sym*
nil
For an ElfXX_Sym* object with name name, print its contents.
getcqctelfsym( edom, esym, name)
edom: domain
esym: ElfXX_Sym*
vector
Convert a raw ELF symbol from the binary into a Cinquecento list of the processed field values of type elfsym.
getelfsecname( edom, ind)
edom: domain
ind: string
string
Get the name of an elf section. This function does not check that there actually is a section at that index.
getelfsection( edom, name)
edom: domain
name: string
string
Get a byte string containing the data of the named ELF section from the provided elf domain (returns "" if no section with that name exists).
getelfsegment( edom, n)
edom: domain
n: string
ElfXX_Phdr*
Return a byte string containing the data of ELF section number n.
getelfsegmenthdr( edom, n)
edom: domain
n: string
ElfXX_Phdr*
Gets header number n from the provided elf domain. Returns null if there is no header n.
getelfsyms(edom)
edom: domain
vector
Returns a vector of cqct symbols in the given elf domain
getsyms(elfdom)
elfdom: domain
table
prints the symbols in a given elf domain, returning a table mapping strings to pointers to ElfXX_Sym objects.
mkelfrec(binfile)
binfile: binary string
bd record
Returns a bd record describing an elf file.
showallelfsectionheaders(elfdom)
elfdom: domain
nil
Prints all sections headers from the given elf domain.
showelfheader(elfdom)
elfdom: domain
nil
Prints the root header from the elf domain.
showelfsym( elfdom, addr, strind )
elfdom: domain
addr: void*
strind: int
nil
Prints a description of the elf symbol at the given addr with the given offset into the e_shoff table.
showsectionheader( elfdom, sbase, index )
elfdom: domain
sbase: elfdom`Elf64_Shdr* or elfdom`Elf32_Shdr*
index: int
nil
Prints a section header at index for the given elf domain.
showsegmentheader( eseg)
elfdom: domain
eseg: elfdom`Elf64_Phdr* or elfdom`Elf32_Phdr*
nil
Print informaiton in an ElfXX_Phdr* object.
showsym(esym)
esym: elfsym
nil
Print information in the given elfsym.
show_elf_addrs(syms)
syms: vector
nil
Prints the address (in hex) for every vector in the list.
show_elf_syms(syms)
syms: vector
nil
Runs showsym over every entry in the given array.
nsgraph.cqct

nsgraph.cqct provides code for creating and analyzing graphs of types inferred from a namespace.

gentypegraph(ns)
ns: namespace
table
Returns a table representation of the graph implied by the structures and unions in the given namespace.
printtypegraph(ns)
ns: namespace
nil
Prints the typegraph associated with the given namespace.
printtypegraph_dot(ns)
ns: namespace
nil
Prints a dot compatible representation of the typegraph associated with the given namespace.
typegraph_edge_iter(tg, fn)
tg: table representation of the typegraph
fn: function
nil
Runs fn(from_node,to_node) for all edges in the typegraph.
typegraph_node_iter(tg, fn)
tg: table representation of the typegraph
fn: function
nil
Runs fn(from_node,to_nodes) for all nodes in the typegraph. Note that the second parameter to fn will be a list.
nsutil.cqct

nsutil.cqct provides code for checking type equality, printing namespaces, and iterating over types in namespaces.

all_types(ns)
ns: namespace
nil
Print all the types in a namespace (not parsable by cqct).
mksnoff(ns, off)
ns: namespace
off: unsigned integer
name space
Returns a namespace whose symbols are are translated off from their original location. For example, the symbol at offset 0x10 in ns1 will be at offset 0x20 in mknsoff(ns1,0x10). This namespace is implemented in such a way as to not interfere with the caching provided by the sctl namespaces.
nsdiff(t1, t2)
n1: type
n2: type
array
Prints the types in each namespace that are not contained in the other namespace or have a different definition in the other namespace. Returns [tab1, tab2, tab3] where tab1 contains types in ns1 that are not in ns2, and tab2 contains types in ns2 that are not in ns1, and tab3 maps types in either ns to a type with the same name in the other ns that is not the same type.
nsiter(ns, fn)
ns: namespace
fn: function
array
Runs fn(type) for every type in the namespace.
nsjoin(ns1, ns2)
ns1: namespace
ns2: namespace
namespace
retuns a namespace that contains all types and symbols in both namespaces ns1 and ns2. When both namespaces define a symbol or a type, the definition in ns2 is preferred.
printns(ns)
ns: namespace
nil
Prints the namespace in a format parsable by cqct in an @names directive.
printtype(t1)
t1: type
nil
Prints the type in a format parsable by cqct.
typeequal(t1, t2)
t1: type
t2: type
int
Recursivly checks that the two types are equal.
typenameequal(t1, t2)
t1: type
t2: type
int
Returns 0 if the type names are different, 1 if they are the same. Does not recursivly check structures but instead checks that they have the same name. Does check that arrays have the same length, functions have the same parameter and return types.
types_with_ptrs(ns)
ns: namespace
nil
Print all the types containing pointers.
rangeset.cqct

rangeset.cqct provides a data structure that stores a set of ranges in a minimal way. The rangeset structure is implemented with a red black tree, and allows one to add and remove ranges with the guarantee that (1) a minimal number of ranges will be stored and (2) checking if a value is contained in a given rangeset will take O(log(n)) time (where n is the number of ranges stored). Example usage:

@include <rangeset.cqct>
rs = rangeset_create();
rangeset_insert(rs,mkrange(50,100));
rangeset_delete(rs,mkrange(60,5));
rangeset_contains(rs,53); //returns 1
rangeset_contains(rs,61); //returns 0
rangeset_enumerate(rs); //returns [ range(50,10), range(65,85) ]
Because ranges only work with non-negative numbers, rangeset only work on non-negative integer ranges.
rangeset_create()
rangeset
Returns an empty rangeset object
rangeset_contains( rs, v )
rs: rangeset
v: integer
int
Returns 1 (true) if there is a range in the given range set that contains the given value v. Returns 0 (false) otherwise.
rangeset_delete( rs, r )
rs: rangeset
r: range
nil
Removes the given range from the given rangeset. The rangeset will no longer "contain" any of the values in the given range. If the rangeset did not contain those values to begin with, then there is no change.
rangeset_enumerate( rs )
rs: rangeset
list
Returns a minimally sized list of ranges that cover exactly the set of values covered by the given rangeset.
rangeset_foreach( rs, fn )
rs: rangeset
fn: function
nil
Runs the given function over a minimal representation of the ranges covered by the given rangeset. That is, fn(r) will be run for a minimal number of r that cover exactly the range covered by this rangeset.
rangeset_insert( rs, r )
rs: rangeset
r: range
nil
Inserts the given range into the given rangeset.
rangeset_intersect( rs, r )
rs: rangeset
r: range
list
Returns a list of all ranges that intersect the given range in the given rangeset.
re.cqct

re.cqct provides an API for using regular expressions in CQCT. Currently, regular expressions containing [], *, ., (), and ? are all supported with the same meanings as in PERL (run "man perlre" for more details), with the following exceptions:

Also as in PERL, \ is an escape character. This regular expression library uses a DFA to allow for matching in time O(n*m), where n is the length of the RE and m is the length of the input. See Russ Cox's Regular Expression Matching for details. Example usage:
@include <re.cqct>
re_match("abc*d","abccccd"); //returns 1
re_match("abc*d","accccd"); //returns 0
re_match("abc*d","xxxabccccd"); //returns 0
re_find("abc*d","xxxabccccd"); //returns 3 
comp = re_create("abc*d"); //pre-compile the re
re_match(comp,"abccccd"); //returns 1
re_match(comp,"accccd"); //returns 0
re_match(comp,"xxxabccccd"); //returns 0
re_find(comp,"xxxabccccd"); //returns 3 
re_create(re)
re: string
DFA
Allows one to create the DFA associated with a given regular expression. These DFA can currently be passed to the re_match or re_find functions. If one is doing multiple matches with the same RE, then it is recommended that one creates the DFA for that RE via this function, rather than forcing re_match or re_find to re-create the DFA on each call.
re_find(re, str)
re: string or DFA
str: string
int
This function take a regular expression (either as a string or as the DFA output from re_create) and a string and returns the first index in the string that matches the regular expression, or -1 if no instances of the expression are found in the given string.
re_findend(re, str [,idx] )
re: string or DFA
str: string
idx: int
int
This function take a regular expression (either as a string or as the DFA output from re_create) and a string and matches the string against the RE, returning the index of the final character in the largest match. So for instance re_findend("aa","aaaa") will return 1 and re_findend("aa*","aaaa") will return 4. If the string is empty and the RE matches, this function will return 0. If the RE does not match, -1 will be returned. If idx is provided, matching will start from the index idx of the given string.
re_match(re, str)
re: string or DFA
str: string
int
This function take a regular expression (either as a string or as the DFA output from re_create) and a string and returns 1 if the regular expression matches the entire string. Otherwise 0 is returned.
rbtree.cqct

rbtree.cqct provides an API for red black trees. Red black trees are balanced and provide O(n) insertion, deletion, and querying.

rbtree_create(compare)
compare: function
rbtree
This function take a comparison function and returns an rbtree object, that can be used in the rbtree_* functions below. The comparison function takes two objects and returns an integer which is negative if the first object is smaller than the second, positive if the first object is larger than the second, and zero if the objects are equal.
rbtree_delete( tree, key)
tree: rbtree
key: keytype
bool
This function removes the key and its associated data from the given rbtree. If multiple associations exist, an arbitrary one is removed. It returns 1 if it was able to successfully remove something, and 0 otherwise.
rbtree_enumerate(tree)
tree: rbtree
list
This function returns a list of all [key,data] pairs in the given tree ordered by key according to the tree's compare function.
rbtree_foreach( tree, fn)
tree: rbtree
fn: function
nil
This function runs the given function fn on each key/data association in the given rbtree. It is required that fn does not modify the given rbtree.
rbtree_insert( tree, key, data)
tree: rbtree
key: keytype
data: anything
nil
This function associates the given key with the given data in the given rbtree.
rbtree_query( tree, key)
tree: rbtree
key: keytype
nil or pair
This function returns the [k,data] associated with the smalest k in the tree that is greater than and not equal to k. If multiple such k exist, then an arbitrary one is returned. If no such k not found in the tree, then nil is returned.
rbtree_query( tree, key)
tree: rbtree
key: keytype
nil or pair
This function returns the [k,data] associated with the largest k in the tree that is less than and not equal to k. If multiple such k exist, then an arbitrary one is returned. If no such k not found in the tree, then nil is returned.
rbtree_print( tree [,fd])
tree: rbtree
fd: file descriptor
nil
This function prints a text version of the rbtree to fd (stdout if fd is not provided). It uses "%a" to print both the key and the data.
rbtree_query( tree, key)
tree: rbtree
key: keytype
nil or anything
This function returns the data associated with the given key in the given rbtree. If multiple data are associated with a given key, then an arbitrary one is returned. If the key is not found in the tree, then nil is returned.
rbtree_rangequery( tree, lower, upper)
tree: rbtree
lower: keytype
upper: keytype
list
This function returns a list of [key,data] pairs stored in the given tree where the key greater than or equal to lower and strictly less than upper.
snames.cqct

snames.cqct is a server intended to serve Cinquecento name spaces across a network using the sctl protocol. It supports two modes of operation: in names mode, name spaces are drawn from a "names files" stored in the local file system; in sctl mode, name spaces are drawn from new invocations of the sctl server (which must be in the path).

The usage is:

snames.cqct port rootns mode prefix [var ...]

Port is the port on which the server accepts connections (from any available network interface).

Rootns is the name of the root name space (e.g., c32le) from which all served name spaces are extended. It is meaningful only in names mode.

Mode is either names or sctl.

Prefix is the path to the directory tree in which the served names files are accessed. To resolve a name space name, the path argument provided by the client snamesns operation is appended to prefix. (If Prefix does not end with the character "/", it is added.) Prefix is meaningful only in names mode.

The remaining optional arguments var ... are each the name of a variable that will be bound, with value 0, in the scope of the @names evaluation that is performed for each served name space. This allows the serving of names files that contain references to variables, but it does not offer any control over their values. These arguments are meaningful only in names mode.

In the event of an error, the snames server automatically restarts. Some diagnostics are printed to standard output.

This command-line synopsis should be revised to not require meaningless arguments in sctl mode.

gdb.cqct

gdb.cqct provides access to an execution target over the GDB remote serial protocol ("GDB stubs").

The library provides a set of functions for directly speaking various messages of the GDB protocol: gdbbrk, gdbc, gdbg, gdbm, gdbM, gdbs, gdbS, gdbz, gdbZ, gdbrecvpkt. This set is likely to grow, and is not documented.

Most users should use the high-level client interface, mkgdbctl:

mkgdbctl(fd)
fd: file descriptor
ctl object
The file descriptor fd should be the client endpoint of a connection to a GDB remote serial protocol server. Returns a control object that supports several methods for accessing the remote target. These methods are interfaces to GDB serial protocol, nothing more: their precise effect depends on the server implementation of the protocol.

The methods are:

cont()
nil
Directs the remote target to resume execution.
stop()
nil
Directs the remote target to stop execution. Returns after the stop-reply packet has been received.
step()
nil
Directs the remote target to single step. Returns upon completion of the step.
wait()
string
Waits for the server to send a stop-reply packet, indicating that the target execution has suspended or exited. Returns the body of the stop-reply packet, from which callers can determine the status of the target.
Z(type,addr,len)
z(type,addr,len)
type: cvalue
addr: cvalue
len: cvalue
string

Sets (Z) or clears (z) a trap on various forms of access to target memory. Type specifies the type of trap, addr specifies the first address of the instrumented range, and len specifies the number of bytes in the range.

The GDB protocol defines five possible values for type:

0 software breakpoint
1 hardware breakpoint
2 memory write watchpoint
3 memory read watchpoint
4 memory access watchpoint
mem()
address space

Returns an address space backed by the address space of the target execution.

BUG: The GDB serial protocol offers visibility into the layout of the target address, but the mem method does not take advantage of it. Instead, it assumes that the target completely maps a 64-bit address space. Any access to memory that is not actually mapped will fail gracelessly.

reg()
string

Returns the current values of the general register set of the target in a contiguous byte array. This array represents the register set values, in binary, returned by the server; the client is responsible for its interpretation.

The following example constructs a gdb control object to the GDB server at TCP port 10000 on the local host, and causes the target to single step ten times:

; @include <gdb.cqct>
; ctl = mkgdbctl(tcpopen("localhost:10000"));
; for(i = 0; i < 10; i++) ctl.step();

This example constructs a domain dom from the control object, using a separately provided name space:

; ns = @names c32le { @include "my.names" };
; dom = mkdom(ns, ctl.mem());