expfactory package¶
Submodules¶
expfactory.battery module¶
battery.py: part of expfactory package Functions to generate batteries
-
expfactory.battery.
add_custom_variables
(custom_variables, template)[source]¶ add_custom_variables takes a list of tuples and a template, where each tuple is a (“[TAG]”,”substitution”) and the template is an open file with the tag. :param custom_variables: a list of tuples (see description above) :param template: an open file to replace “tag” with “substitute”
-
expfactory.battery.
generate
(battery_dest=None, battery_repo=None, experiment_repo=None, experiments=None, config=None, make_config=True, warning=True, time=30)[source]¶ will create a battery from a template and list of experiments :param battery_dest: is the output folder for your battery. This folder MUST NOT EXIST. If not specified, a temp folder is created :param battery_repo: location of psiturk-battery repo to use as a template. If not specified, will be downloaded to a temporary directory :param experiment_repo: location of a expfactory-experiments repo to check for valid experiments. If not specified, will be downloaded to a temporary directory :param experiments: a list of experiments, meaning the “exp_id” variable in the config.json, to include. This variable also conincides with the experiment folder name. :param config: A dictionary with keys that coincide with parameters in the config.txt file for a expfactory experiment. If not provided, a dummy config will be generated. :param make_config: A boolean (default True) to control generation of the config. If there is a config generated before calling this function, this should be set to False. :param warning: Show config.json warnings when validating experiments. Default is True :param time: maximum amount of time for battery to endure (default 30 minutes) to select experiments
-
expfactory.battery.
generate_base
(battery_dest, tasks=None, experiment_repo=None, survey_repo=None, game_repo=None, add_experiments=True, add_surveys=True, add_games=True, battery_repo=None, warning=True)[source]¶ generate_base returns a folder with downloaded experiments, surveys, and battery, either specified by the user or a temporary directory, to be used by generate_local and generate (for psiturk) :param battery_dest: [required] is the output folder for your battery. This folder MUST NOT EXIST. :param battery_repo: location of psiturk-battery repo to use as a template. If not specified, will be downloaded to a temporary directory :param experiment_repo: location of a expfactory-experiments repo to check for valid experiments. If not specified, will be downloaded to a temporary directory :param survey_repo: location of a expfactory-surveys repo to check for valid surveys. If not specified, will be downloaded to a temporary directory :param tasks: a list of experiments and surveys, meaning the “exp_id” variable in the config.json, to include. This variable also conincides with the tasks folder name. :param warning: show warnings when validating experiments (default True)
-
expfactory.battery.
generate_config
(battery_dest, fields)[source]¶ takes a dictionary, and for matching fields, substitues and prints to “config.txt” in a specified battery directory :param battery_dest: should be the copied, skeleton battery folder in generation :param fields: should be a dictionary with fields that match those in the config non matching fields will be ignored.
-
expfactory.battery.
generate_local
(battery_dest=None, subject_id=None, battery_repo=None, experiment_repo=None, experiments=None, warning=True, time=30)[source]¶ generate_local deploys a local battery will create a battery from a template and list of experiments :param battery_dest: is the output folder for your battery. This folder MUST NOT EXIST. If not specified, a temp directory will be used :param battery_repo: location of psiturk-battery repo to use as a template. If not specified, will be downloaded to a temporary directory :param experiment_repo: location of a expfactory-experiments repo to check for valid experiments. If not specified, will be downloaded to a temporary directory :param experiments: a list of experiments, meaning the “exp_id” variable in the config.json, to include. This variable also conincides with the experiment folder name. :param subject_id: The subject id to embed in the experiment, and the name of the results file. If none is provided, a unique ID will be generated. :param time: Maximum amount of time for battery to endure, to select experiments
-
expfactory.battery.
get_concat_js
(valid_experiments)[source]¶ Return javascript concat section for valid experiments, based on psiturk.json :param valid_experiments: full paths to valid experiments to include
..note:
case "simple-rt": experiments = experiments.concat(simple-rt_experiment) break; case "choice-rt": experiments = experiments.concat(choice-rt_experiment) break; Format for experiment variables is [exp_id]_experiment
-
expfactory.battery.
get_experiment_run
(valid_experiments, deployment='local')[source]¶ returns a dictionary of experiment run code (right now just jspsych init objects) :param valid_experiments: full path to valid experiments folders, OR a loaded config.json (dict)
-
expfactory.battery.
get_load_js
(valid_experiments, url_prefix='')[source]¶ Return javascript to load list of valid experiments, based on psiturk.json :param valid_experiments: a list of full paths to valid experiments to include
- ..note::
- Format is:
- {
- case “simple_rt”:
- loadjscssfile(“static/css/experiments/simple_rt.css”,”css”) loadjscssfile(“static/js/experiments/simple_rt.js”,”js”) break;
- case “choice_rt”:
- loadjscssfile(“static/css/experiments/choice_rt.css”,”css”) loadjscssfile(“static/js/experiments/choice_rt.js”,”js”) break;
… }
-
expfactory.battery.
get_load_static
(valid_experiments, url_prefix='', unique=True)[source]¶ return the scripts and styles as <link> and <script> to embed in a page directly :param unique: return only unique scripts [default=True]
-
expfactory.battery.
get_timing_js
(valid_experiments)[source]¶ Produce string (json / dictionary) of experiment timings :param valid_experiments: a list of full paths to valid experiments to include
..note:
Produces the following format for each experiment {name:"simple_rt", time: 3.5}, {name:"choice_rt", time: 4}, ...
-
expfactory.battery.
move_experiments
(valid_experiments, battery_dest, repo_type='experiments')[source]¶ Moves valid experiments into the experiments folder in battery repo :param valid_experiments: a list of full paths to valid experiments :param battery_dest: full path to battery destination folder :param repo_type: the kind of task to move (default is experiments)
-
expfactory.battery.
template_experiments
(battery_dest, battery_repo, valid_experiments, template_load=None, template_exp=None, template_exp_output=None, custom_variables=None)[source]¶ template_experiments: For each valid experiment, copies the entire folder into the battery destination directory, and generates templates with appropriate paths to run them :param battery_dest: full path to destination folder of battery :param battery_repo: full path to psiturk-battery repo template :param valid_experiments: a list of full paths to experiment folders to include :param template_load: the load_experiments.js template file. If not specified, the file from the battery repo is used. :param template_exp: the exp.html template file that runs load_experiment.js. If not specified, the psiturk file from the battery repo is used. :param template_exp_output: The output file for template_exp. if not specified, the default psiturk templates/exp.html is used :param custom_variables: A dictionary of custom variables to add to templates. Keys should either be “exp” or “load”, and values should be tuples with the first index the thing to sub (eg, [SUB_THIS_SUB]) and the second the substitition to make.
expfactory.experiment module¶
experiment.py: part of expfactory package Functions to work with javascript experiments
-
expfactory.experiment.
check_acceptable_variables
(experiment_name, field_dict, template, field_dict_key)[source]¶ check_acceptable_variables takes a field (eg, meta[0][field]) that has a dictionary, and some template key (eg, jspsych) and makes sure the keys of the dictionary are within the allowable for the template type (the key). :param experiment_name: the name of the experiment :param field_dict: the field value from the config.json, a dictionary :param field_dict_key: a key to look up in the field_dict, which should contain a dictionary of {“key”:”value”} variables :param template: the key name, for looking up acceptable values using get_acceptable_values
-
expfactory.experiment.
check_boolean
(experiment_name, value, variable_name)[source]¶ check_boolean checks if a value is boolean :param experiment_name: the name of the experiment :param value: the value to check :param variable_name: the name of the variable (the key being indexed in the dictionary)
-
expfactory.experiment.
find_changed
(new_repo, comparison_repo, return_experiments=True, repo_type='experiments')[source]¶ find_changed returns a list of changed files or experiments between two repos :param new_repo: the updated repo - any new files, or changed files, will be returned :param comparison_repo: the old repo to compare against. A file changed or missing in this repo in the new_repo indicates it should be tested :param return_experiments: return experiment folders. Default is True. If False, will return complete file list
-
expfactory.experiment.
get_experiments
(experiment_repo, load=False, warning=True, repo_type='experiments')[source]¶ return loaded json for all valid experiments from an experiment folder :param experiment_repo: full path to the experiments repo :param load: if True, returns a list of loaded config.json objects. If False (default) returns the paths to the experiments :param repo_type: tells the user what kind of task is being parsed, default is “experiments,” but can also be “surveys” when called by get_surveys
-
expfactory.experiment.
get_validation_fields
()[source]¶ Returns a list of tuples (each a field)
..note:
specifies fields required for a valid json (field,value,type) field: the field name value: indicates minimum required entires 0: not required, no warning 1: required, not valid 2: not required, warning type: indicates the variable type
-
expfactory.experiment.
load_experiment
(experiment_folder)[source]¶ load_experiment: reads in the config.json for an :param experiment folder: full path to experiment folder
-
expfactory.experiment.
load_experiments
(experiment_folders)[source]¶ a wrapper for load_experiment to read multiple experiments :param experiment_folders: a list of experiment folders to load, full paths
-
expfactory.experiment.
make_lookup
(experiment_list, key_field)[source]¶ returns dict object to quickly look up query experiment on exp_id :param experiment_list: a list of query (dict objects) :param key_field: the key in the dictionary to base the lookup key (str) :returns lookup: dict (json) with key as “key_field” from query_list
-
expfactory.experiment.
validate
(experiment_folder=None, warning=True)[source]¶ Parameters: - experiment_folder – full path to experiment folder with config.json
- warning – issue a warning for empty fields with level 2 (warning)
..note:
takes an experiment folder, and looks for validation based on: - config.json - files existing specified in config.json All fields should be defined, but for now we just care about run scripts
expfactory.interface module¶
expfactory.utils module¶
utils.py: part of expfactory package
-
expfactory.utils.
find_directories
(root, fullpath=True)[source]¶ Return directories at one level specified by user (not recursive)
-
expfactory.utils.
find_subdirectories
(basepath)[source]¶ Return directories (and sub) starting from a base
-
expfactory.utils.
get_template
(template_file)[source]¶ get_template: read in and return a template file
-
expfactory.utils.
is_type
(var, types=[<type 'int'>, <type 'float'>, <type 'list'>])[source]¶ Check type
expfactory.scripts module¶
script.py: part of expfactory api Runtime executable
expfactory.views module¶
views.py
part of the experiment factory package functions for developing experiments and batteries, viewing and testing things
-
expfactory.views.
embed_experiment
(folder, url_prefix='')[source]¶ return an html snippet for embedding into an application. This assumes the same directory structure, that all jspsych files can be found in static/js/jspych, and experiments under static/experiments/[folder] :param folder: full path to experiment folder with config.json
-
expfactory.views.
generate_experiment_web
(output_dir, experiment_folder=None, survey_folder=None, games_folder=None, make_table=True, make_index=True, make_experiments=True, make_data=True, make_surveys=True, make_games=True)[source]¶ get_experiment_table Generate a table with links to preview all experiments :param experiment_folder: folder with experiments inside :param survey_folder: folder with surveys inside :param games_folder: folder with games inside :param output_dir: output folder for experiment and table html, and battery files :param make_table: generate table.html :param make_index: generate d3 visualization index :param make_experiments: generate experiment preview files (linked from table and index) :param make_data: generate json/tsv data to download :param make_surveys: generate static files for surveys repos :param make_games: generate static files for games repos
-
expfactory.views.
get_cognitiveatlas_hierarchy
(experiment_tags=None, get_html=False)[source]¶ return :param experiment_tags: a list of experiment exp_id tags to include. If None provided, all valid experiments will be used. :param get_html: if True, returns rendered HTML template with hierarchy. False returns json data structure.
-
expfactory.views.
get_experiment_html
(experiment, experiment_folder, url_prefix='', deployment='local')[source]¶ return the html template to test a single experiment :param experiment: the loaded config.json for an experiment (json) :param experiment_folder: the experiment folder, needed for reading in a survey :param url_prefix: prefix to put before paths, in case of custom deployment :param deployment: deployment environment, one of docker, docker-preview, or local [default]
-
expfactory.views.
preview_experiment
(folder=None, battery_folder=None, port=None)[source]¶ preview an experiment locally with the –preview tag (for development) :param folder: full path to experiment folder to preview. If none specified, PWD is used :param battery_folder: full path to battery folder to use as a template. If none specified, the expfactory-battery repo will be used. :param port: the port number, default will be randomly generated between 8000 and 9999 :param robot: if True, a web server is started as a separate process for a robot to run
-
expfactory.views.
run_battery
(destination=None, experiments=None, experiment_folder=None, subject_id=None, battery_folder=None, port=None, time=30)[source]¶ run_battery runs or previews an entire battery locally with the –run tag. If no experiments are provided, all in the folder will be used. :param destination: destination folder for battery. If none provided, tmp directory is used :param experiments: list of experiment tags to add to battery :param experiment_folder: the folder of experiments to deploy the battery from. :param subject_id: subject id to embed into battery. If none, will be randomly generated :param battery_folder: full path to battery folder to use as a template. If none specified, the expfactory-battery repo will be used. :param port: the port number, default will be randomly generated between 8000 and 9999 :param time: total number of minutes for experiments to add to battery
-
expfactory.views.
run_single
(exp_id, repo_type, destination=None, source_repo=None, battery_repo=None, port=None, subject_id=None)[source]¶ run_survey runs or previews an entire battery locally with the –run tag. If no experiments are provided, all in the folder will be used. :param destination: destination folder for battery. If none provided, tmp directory is used :param exp_id: exp_id for survey, experiment, or game to run (unique ID) :param repo_type: must be within experiments,surveys, or games :param source_repo: a source repo for the experiment, survey, or game :param subject_id: subject id to embed into result. If none, will be randomly generated :param battery_folder: full path to battery folder to use as a template. If none specified, the expfactory-battery repo will be used. :param port: the port number, default will be randomly generated between 8000 and 9999
-
expfactory.views.
tmp_experiment
(folder=None, battery_folder=None)[source]¶ generate temporary directory with experiment :param folder: full path to experiment folder to preview (experiment, survey, or game). If none specified, PWD is used :param battery_folder: full path to battery folder to use as a template. If none specified, the expfactory-battery repo will be used.
expfactory.vm module¶
vm.py: part of expfactory package Functions to generate virtual machines to run expfactory batteries
-
expfactory.vm.
add_custom_logo
(battery_repo, logo)[source]¶ Add a custom logo to the vm battery :param battery_repo: the full path to the battery repo base, assumed to have an “img” folder :param logo: the full path to the logo to copy. Should ideally be png.
-
expfactory.vm.
custom_battery_download
(tmpdir=None, repos=['experiments', 'battery', 'surveys', 'games'])[source]¶ Download battery and experiment repos to a temporary folder to build a custom battery, return the path to the tmp folders :param tmpdir: The directory to download to. If none, a temporary directory will be made :param repos: The repositories to download, valid choices are “experiments” “battery” and “vm”
-
expfactory.vm.
download_repo
(repo_type, destination)[source]¶ Download a expfactory infrastructure repo “repo_type” to a “destination” :param repo_type: can be one of “experiments” “battery” “vm” :param destination: the full path to the destination for the repo
-
expfactory.vm.
generate_database_url
(dbtype=None, username=None, password=None, host=None, table=None, template=None)[source]¶ Generate a database url from input parameters, or get a template :param dbtype: the type of database, must be one of “mysql” or “postgresql” :param username: username to connect to the database :param password: password to connect to the database :param host: database host :para table: table in the database :param template: can be one of “mysql” “sqlite3” or “postgresql” If specified, all other parameters are ignored, and a default database URL is produced to work in a Vagrant VM produced by expfactory
-
expfactory.vm.
get_jspsych_init
(experiment, deployment='local', finished_message=None)[source]¶ return entire jspsych init structure :param experiment: the loaded config.json for the experiment :param deployment: specify to deploy local (default), or docker-mturk,docker-local (expfactory-docker). :param finished_message: custom message to show at the end of the experiment with Redo and Next Experiment Buttons
-
expfactory.vm.
get_stylejs
(experiment, url_prefix='')[source]¶ return string for js and css scripts to insert into a page based on battery path structure
-
expfactory.vm.
prepare_vm
(battery_dest, fields=None, vm_repo=None, vm_type='vagrant')[source]¶ Prepare virtual machine to run local with vagrant, or with vagrant-aws :param battery_dest: the battery destination folder :param fields: if specified, will be added to the config.txt :param vm_repo: the expfactory-vm repo to use for templates. If not provided, one will be downloaded to a temporary directory for use. :param vm_type: one of “vagrant” or “aws” Default is “vagrant” meaning a localvirtual machine with vagrant.