dbfun Package

dbfun Package

dbfun Module

Abstract API for getting functions of data in a database

API define one attribute (input_names) and one method (evaluate)

class ABXpy.dbfun.dbfun.DBfun(input_names)

Bases: object

evaluate(inputs_dict)

output_specs()

dbfun_column Module

Created on Fri Dec 20 13:36:52 2013

@author: Thomas Schatz

class ABXpy.dbfun.dbfun_column.DBfun_Column(name, db=None, column=None, indexed=True)

Bases: ABXpy.dbfun.dbfun.DBfun

evaluate(context)

output_specs()

dbfun_compute Module

Created on Mon Oct 14 16:59:27 2013

@author: Thomas Schatz

Class for defining and computing efficiently functions of the columns of a database. Implements the DBfun API

class ABXpy.dbfun.dbfun_compute.DBfun_Compute(definition, columns)

Bases: ABXpy.dbfun.dbfun.DBfun

evaluate(context)

output_specs()

parse()

first separate the script defining the function into various components (import statements, with ‘h5file’ statement, synopsis definition, main code)

process_with(tree)

class ABXpy.dbfun.dbfun_compute.nameVisitor(*args, **kwargs)

Bases: ast.NodeVisitor

visit_Name(node)

dbfun_lookuptable Module

Created on Mon Nov 25 00:24:42 2013

@author: Thomas Schatz

Implements the DBfun API Basic implementation of database function in lookup tables. Allows on-the-fly computation by storing script for DBfun_compute alongside the table. Allows to store outputs with h5 compatible dtypes either directly or under an indexed format Do not implement variable length outputs and requires that the entire lookup table fits in RAM memory.

class ABXpy.dbfun.dbfun_lookuptable.DBfun_LookupTable(filename, synopsis=None, db=None, code=None, indexed=True)

Bases: ABXpy.dbfun.dbfun.DBfun

compress_index(indexed_output_name)

evaluate(context)

fill(data, append=False, iterate=False)

get_keys(input_values)

initialize_output_dsets(sample_data)

load()

load_data()

load_metadata()

output_specs()

pack()

write(data)

ABXpy.dbfun.dbfun_lookuptable.chunk_size(item_size=4, n_columns=1, size_in_mem=400)

ABXpy.dbfun.dbfun_lookuptable.get_dtype(data)

lookuptable_connector Module

AST visitor class for finding calls to lookup table (.dbfun) files in a script and make the script ready for execution.

There is a restriction: when in a lookup table calling node, check that the node.code_nodes concatened of the children is empty: i.e. hierarchical calls to auxiliary h5 file are not allowed with a depth larger than 1. We could allow deeper calls, but would there be any practical benefit ? Probably not because the input to a lookup table call can only be of the type of a column of the database (at least with the current system).

class ABXpy.dbfun.lookuptable_connector.LookupTableConnector(script, aux_functions, aliases, *args, **kwargs)

Bases: ast.NodeTransformer

bundle_call_info(node, call_node)

check_Call(node)

check_Subscript(node, func_name)

check_flatness(node)

generic_visit(node)

Called if no explicit visitor function exists for a node.

visit_Call(node)

visit_Subscript(node)