How to contribute

First off, thanks for taking the time to contribute to The COBRA Toolbox!

You can install the MATLAB.devTools from within MATLAB by typing:

>> installDevTools()

Check out the MATLAB.devTools - and contribute the smart way! The official documentation is here.

Contribute to the opencobra/cobratoolbox repository by following these instructions:

>> contribute('opencobra/cobratoolbox');

Contribute to the opencobra/COBRA.tutorials repository by following these instructions:

>> contribute('opencobra/COBRA.tutorials');

• Please follow the Style Guide.
• More information on writing a test is here and a template is here.
• More information on formatting the documentation is here
• A guide for reporting an issue is here.

If you want to use git via the command line interface and need help, this guide or the official GitHub guide come in handy.

Guide for writing a test

Before starting to write a test on your own, it might be instructive to follow common test practices in /test/verifiedTests. A style guide on how to write tests is given here.

Prepare the test (define requirements)

There are functions that need a specific solver or that can only be run if a certain toolbox is installed on a system. To address these, you should specify the respective requirements by using

solvers = prepareTest(requirements)

If successfull and all requirements are fulfilled, prepareTest will return a struct with one field for each problem type (solvers.LP, solvers.MILP etc.). Each field will be a cell array of solver names (if any are available). If the test does not ask for multiple solvers (via the requiredSolvers or the useSolversIfAvailable arguments), the returned cell array will only contain at most one solver.

Here are a few examples:

Example A: Require Windows for the test

The same works with needsMac, needsLinux and needsUnix instead of needsWindows.

solvers = prepareTest('needsWindows', true);

Example B: Require an LP solver (needsLP)

The same works with needsNLP, needsMILP, needsQP or needsMIQP. solvers.LP, solvers.MILP etc. will be cell arrays of string with the respective available solver for the problem type. If the 'useSolversIfAvailable' parameter is non empty, all installed solvers requested will be in the cell array. Otherwise, there will be at most one solver (if no solver for the problem is installed, the cell array is empty).

solvers = prepareTest('needsLP', true);

Example C: Use multiple solvers if present

If multiple solvers are requested. solvers.LP, solvers.MILP etc will contain all those requested solvers that can solve the respective problem type and that are installed.

solvers = prepareTest('needsLP', true, 'useSolversIfAvailable', {'ibm_cplex', 'gurobi'});

Example D: Require one of a set of solvers

Some functionalities do only work properly with a limited set of solvers. with the keyword requireOneSolverOf you can specify a set of solvers of which the test requires at least one to be present. This option, will make the test never be run with any but the solvers specified in the supplied list. E.g. if your test only works with gurobi or mosek you would call prepareTest as

solvers = prepareTest('requireOneSolverOf', {'ibm_cplex', 'gurobi'})

Example E: Exclude a solver

If for some reason, a function is known not to work with a few specific solvers (e.g. precision is insufficient) but works with all others, it might be advantageous to explicitly exclude that one solver instead of defining the list of possible solvers. This can be done with the excludeSolvers parameter. Eg. to exclude matlab and lp_solve you would use the following command:

solvers = prepareTest('excludeSolvers', {'matlab', 'lp_solve'})

Example F: Require multiple solvers

Some tests require more than one solver to be run, and otherwise fail. To require multiple solvers for a test use the requiredSolvers parameters. E.g. if your function requires ibm_cplex and gurobi use the following call:

solvers = prepareTest('requiredSolvers', {'ibm_cplex', 'gurobi'})

Example G: Require a specific MATLAB toolbox

The toolbox IDs are specified as those used in license('test', 'toolboxName'). The following example requires the statistics toolbox to be present.

solvers = prepareTest('requiredToolboxes', {'statistics_toolbox'})

Example H: Multiple requirements

If the test requires multiple different properties to be met, you should test them all in the same call. To keep the code readable, first define the requirements and then pass them in.

% define required toolboxes
requiredToolboxes = {'bioinformatics_toolbox', 'optimization_toolbox'};

% define the required solvers (in this case matlab and dqqMinos)
requiredSolvers = {'dqqMinos', 'matlab'};

% check if the specified requirements are fullfilled (toolboxes, solvers in thhis example, a unix OS).
solversPkgs = prepareTest('requiredSolvers', requiredSolvers, 'requiredToolboxes', requiredToolboxes, 'needsUnix', true);

Test if an output is correct

If you want to test if the output of a function [output1, output2] = function1(input1, input2) is correct, you should call this function at least 4 times in your test. The argument ìnput2 might be an optional input argument.

% Case 1: test with 1 input and 1 output argument
output1 = function1(input1)

% Case 2: test with 1 input and 2 output arguments
[output1, output2] = function1(input1)

% Case 3: test with 1 output and 2 input arguments
output1 = function1(input1, input2)

% Case 4: test with 2 input and 2 output arguments
[output1, output2] = function1(input1, input2)

Each of the 4 test scenarios should be followed by a test on output1 and output2. For instance, for Case 4:

% Case 4: test with 2 input and 2 output arguments
[output1, output2] = function1(input1, input2)

% test on output1
assert(output1 < tol); % tol must be defined previously, e.g. tol = 1e-6;

% test on output2
assert(abs(output2 - refData_output2) < tol); % refData_output2 can be loaded from a file

The test succeeds if the argument of assert() yields a true logical condition.

Test if a function throws an error or warning message

If you want to test whether your function1 correctly throws an error message, you can test as follows:

% Case 5: test with 2 input and 1 output arguments (2nd input argument is of wrong dimension)
% There are two options. If a particular error message is to be tested (here, 'Input2 has the wrong dimension'):
assert(verifyCobraFunctionError('function1', 'inputs', {input1, input2'}, 'testMessage', 'Input2 has the wrong dimension'));

% If the aim is to test, that the function throws an error at all
assert(verifyCobraFunctionError('function1', 'inputs', {input1, input2'}));

If you want to test whether your function1 correctly throws a warning message, you can test as follows:

warning('off', 'all')
    output1 = function1(input1, input2');
    assert(length(lastwarn()) > 0)
warning('on', 'all')

Note that this allows the error message to be thrown without failing the test.

Test template

A test template is readily available here. The following sections shall be included in a test file:

1. Header

% The COBRAToolbox: <testNameOfSrcFile>.m
%
% Purpose:
%     - <provide a short description of the purpose of the test
%
% Authors:
%     - <major change>: <your name> <date>
%

2. Test initialization

global CBTDIR

% save the current path and switch to the test path
currentDir = cd(fileparts(which('fileName')));

% get the path of the test folder
testPath = pwd;

3. Define the solver packages to be tested and the tolerance

% set the tolerance
tol = 1e-8;

% define the solver packages to be used to run this test
solvers = prepareTest('needsLP',true);

4. Load a model and/or reference data

% load a model distributed by the toolbox
getDistributedModel('testModel.mat');
% load a particular model for this test:
readCbModel([testPath filesep 'SpecificModel.mat'])
% load reference data
load([testPath filesep 'testData_functionToBeTested.mat']);

Please only load small models, i.e. less than 100 reactions. If you want to use a non-standard test model that is already available online, please make a pull request with the URL entry to the COBRA.models repository.

warning:In order to guarantee compatibility across platforms, please use the full path to the model. For instance:

5. Create a parallel pool

This is only necessary for tests that test a function that runs in parallel.

% create a parallel pool
poolobj = gcp('nocreate'); % if no pool, do not create new one.
if isempty(poolobj)
    parpool(2); % launch 2 workers
end

warning:Please only launch a pool of 2 workers - more workers

should not be needed to test a parallel function efficiently.

6. Body of test

The test itself. If the solvers are essential for the functionality tested in this test use:

for k = 1:length(solvers.LP)
    fprintf(' -- Running <testFile> using the solver interface: %s ... ', solvers.LP{k});

    solverLPOK = changeCobraSolver(solvers.LP{k}, 'LP', 0);
    % <your test goes here>

    % output a success message
    fprintf('Done.\n');
end

This is important, as the continuous integration system will run other solvers on the test in its nightly build. That way, we can determine solvers that work with a specific method, and those that do not (potentially due to precision problems or other issues). If the solvers are only used to test the outputs of a function for correctness, use:

solverLPOK = changeCobraSolver(solvers.LP{1}, 'LP', 0);
% <your test goes here>

% output a success message
fprintf('Done.\n');

7. Return to the original directory

% change the directory
cd(currentDir)

Run the test locally on your machine

Please make sure that your test runs individually by typing after a fresh start:

>> initCobraToolbox
>> <testName>

Please then verify that the test runs in the test suite by running:

>> testAll

Alternatively, you can run the test suite in the background by typing:

$ matlab -nodesktop -nosplash < test/testAll.m

Verify that your test passed

Once your pull request (PR) has been submitted, you will notice an orange mark next to your latest commit. Once the continuous integration (CI) server succeeded, you will see a green check mark. If the CI failed, you will see a red cross.

What should I do in case my PR failed?

You can check why your PR failed by clicking on the mark and following the respective links. Alternatively, you can see the output of the CI for your PR here. You can then click on the build number. Under Console Output, you can see the output of test/testAll.m with your integrated PR.

Once you understood why the build for your proposed PR failed, you can add more commits that aim at fixing the error, and the CI will be re-triggered.

Common errors include:

• Double percentage sign %% in your test file to separate code blocks. Replace %% with %.
• Compatibility issues (ILOG Cplex is not compatible with R2015b+). Add an additional test on the version of matlab using verLessThan('matlab', '<version>').

Can I find out how many tests have failed?

The logical conditions, when tested using assert(), will throw an error when not satisfied. It is bad practice to test the sum of tests passed and failed. Please only test using assert(logicalCondition). Even though a test may fail using assert(), a summary table with comprehensive information is provided at the end of the test run.

For instance, the following test script do not do this - bad practice!:

% do not do this: bad practice!
testPassed = 0;
testFailed = 0;

% test on logical condition 1 - do not do this: bad practice!
if logicalCondition1
    testPassed = testPassed + 1;
else
    testFailed = testFailed + 1;
end

% test on logical condition 2 - do not do this: bad practice!
if logicalCondition2
    testPassed = testPassed + 1;
else
    testFailed = testFailed + 1;
end

assert(testPassed == 2 && testFailed == 0); % do not do this: bad practice!

shall be rewritten as follows:

% good practice
assert(logicalCondition1);
assert(logicalCondition2);

Documentation guide

In order to enable the automatic documentation generation, the header of a MATLAB function has to be formatted properly. The automatic documentation extracts the commented header of the function (commented lines between the function’s signature and the first line of code) based on keywords and blocks.

Rule 1: There should be 1 free space between the percentage sign % and the text. For instance, a correctly formatted line in the header reads:

% this is correct

A line in the header that is formatted as follows is ignored:

%this is not correct (note the space)

Rule 2: After the last line of the header, leave one empty line before the body of the function.

% This is the end of the header of the function

x = 5;  % the body of the function begins after one empty line

Importantly, do not put a comment above the first line of code, but include the comment inline:

% This is the end of the header of the function

% the body of the function begins after one empty line
x = 5;

Rule 3: A line in the header that includes .. after the percentage sign will be ignored in the documentation:

% .. this line will be ignored

Function signature

The function signature must be correctly formatted. Leave a space after every comma and before and after the equal sign =. A correctly formatted function signature reads:

function [output1, output2, output3] = someFunction(input1, input2, input3) % good practice

A function signature that is not formatted properly throws an error during the automatic documentation generation:

function [ output1,output2,output3 ]=someFunction( input1,input2,input3 ) % bad practice

Function description

The description of the function is a brief explanation of the purpose of the function. The description of the function may extend over several lines. However, try to keep the explanations as brief as possible.

function [output1, output2, output3] = someFunction(input1, input2, input3)
% This is a description of the function that helps to understand how the function works
% Here the description continues, then we leave an empty comment line
%

Keywords

The automatic documentation software relies on keywords to properly format the documented function. A keyword also defines the start of a block with the header of a function. Main keywords include:

• USAGE:: block for defining how to use the function
• INPUT: or INPUTS:: block with input argument(s)
• OUTPUT: or OUTPUTS:: block with output argument(s)
• EXAMPLE:: block with example code (formatted MATLAB syntax)
• NOTE:: highlighted box with text
• Author:: list with author(s)

Each of them must be followed by non-empty lines and should be separated from the next block by an empty line.

All keywords are optional. For instance, if a function does not have any input arguments, the keyword INPUTS: can be omitted.

Any line of the block must be indented by 4 spaces after the comment sign %:

% INPUTS:
%    input1:     Description of input1
%    input2:     Description of input2
% input3:    Description <-- this is bad practice
%
% OUTPUTS:
%    output1:    Description of output1

If the indentation differs, there will be an error.

Keyword USAGE:

In the block starting with the keyword USAGE:, the function’s signature must be given in order to show how the function should be used. It is important to leave one empty line before the keyword USAGE:, after the keyword, and after the function’s signature.

% the end of the description
%
% USAGE:
%
%    [output1, output2, output3] = someFunction(input1, input2, input3)
%
% here the other section can begin

Keyword INPUT: and OUTPUT:

The arguments declared in the blocks: INPUT:, INPUTS:, OUTPUT: and OUTPUTS: must be followed by a colon : before the argument description is provided.

The indentation between the argument (with colon) and the description should be at least 4 spaces, so that all argument descriptions are aligned.

% INPUTS:
%    input1:     Description of input1 <-- good practice
%    input2      No colon <-- bad practice
%    input3: Not enough distance (4+ spaces) <-- bad practice
%
% OUTPUTS:
%    longerNameOutput:    Description of longerNameOutput after 4 spaces
%    output1:             Description begins at the same place as the longest argument <-- good practice
%    output2:    Description begins too soon <-- bad practice

For a structure argument, it is possible to list its fields. An empty line is added after the structure argument. Then, in the next line, the field is written aligned with the description of the structure plus 2 extra spaces.

The field is listed beginning as * .field - description (note the space between * and .). It is not necessary to leave an empty line after listing fields and writing the next argument. The following illustrates how to list a structure with its fields:

% OUTPUT:
%    output:    output argument with fields:
%
%                 * .field1 - first field of the structure.
%               * .field2 - no indent <-- bad practice
%                 * .field3 - multi-line comment must begin always
%                   where the text of the first line begins <-- good practice
%                 * .field4 - multi-line comment where
%                 the text in line 2 begins too soon <-- bad practice
%    next:      next argument can be added without empty line

It is also possible to replace * with a numbered list. You can use numbers followed by a dot (e.g., 1.) instead of * ..

% OPTIONAL INPUT:
%    input:    optional input argument with fields:
%
%                1. first element of a numbered list
%                2. second element of a numbered list

Keyword EXAMPLE:

A common usage example can be included in the EXAMPLE: block. Code included in this block will be formatted as MATLAB formatted code. Leave one empty line before the keyword EXAMPLE:, after the keyword, and after the properly indented (4 spaces) code snippet.

% the previous block ends here
%
% EXAMPLE:
%
%    result = someFunction(input1, input2)
%    %additional comment if necessary
%
% another block begins here

Keyword NOTE:

Important information, such as common errors, can be included in the block that starts with the keyword NOTE:. A NOTE: block is formatted in a dedicated and highlighted box in the documentation.

Leave one empty line before the keyword NOTE:, after the keyword, and after the properly indented text (indent of 4 spaces).

Normally formatted text can be left at the with one space after the comment sign. An example of a NOTE: block reads:

%
% NOTE:
%
%    This is a note that contains a important information.
%    It will be clearly visible in the documentation online.
%
% This is an additional final comment that can be added and that is
% only relevant to the code itself

Keyword Author: or Author(s):

In the Author(s) block, the author(s) that have written or contributed to the function are listed. Authors are not shown in the documentation itself, so the keyword is preceded by ... List 1 author as follows:

%
% .. Author: - Name, date, additional information if needed

x = 5;  % here the body of the function begins

If there are 2 or more authors, format as follows:

%
% .. Authors:
%       - Name1, date, additional information if needed
%       - Name2, date, additional information if needed

x = 5;  % here the body of the function begins

Example

A complete example of a function is provided here. Please remember that colons, indentations, and keywords are important to guarantee pretty formatting.

function [output1, output2] = someFunction(input1, input2, input3, input4)
% This is a description of the function that helps understand how the function works
% Here the description continues, then we leave an empty comment line
%
% USAGE:
%
%    [output1, output2] = someFunction(input1, input2, input3, input4)
%
% INPUTS:
%    input1:     Description of input1
%    input2:     Description of input2
%
% OPTIONAL INPUTS:
%    input3:     Structure with fields:
%
%                       * First field - description
%                       * Second field - description
%    input4:     Description of input4
%
% OUTPUT:
%    output1:    Description of output1
%
% OPTIONAL OUTPUT:
%    output2:    Description of output2
%
% EXAMPLE:
%
%    % this could be an example that can be copied from the documentation to MATLAB
%    [output1, output2] = someFunction(11, '22', structure, [1;2])
%    % without optional values
%    output1 = someFunction(11, '22')
%
% NOTE:
%    This is a very important information to be highlighted
%
% This is a final comment that cannot be in the description but can be useful
%
% .. Author: - Name, date, some information

x = 5;  % here the body of the function begins

Style guide

A comprehensive MATLAB style guide written by Richard Jonson can be found [here](http://www.datatool.com/downloads/MatlabStyle2%20book.pdf).

Code

1. Spacing

• Write if singleCondition, and not if (singleCondition). Use brackets only for multiple conditions.
• Use spaces around operators, e.g., i + 1 instead of i+1
• Use spaces after commas (unless separated by newlines)
• Avoid spaces inside the curly-braces of cells: {a, b} instead of { a, b }
• Use spaces after commas in lists, after operands, after names, etc. This also improves readability. e.g. a = [1, 2, 3; 4, 5, 6]; instead of a=[1,2,3;4,5,6];
• Include a single line of whitespace between blocks of code
• Include a whitespace after a comment sign %

2. Variable names

• When using mixed words, separate with capital letters (with no intervening spaces or punctuation), e.g. calculateKineticFlux
• Avoid ambiguity when naming variables: be as specific as possible
• All variable names must be written in English
• Use verb-noun structure for functions: allows to explain the operations performed
• Append meaningful prefixes when possible, e.g. Av, Sum, Min, Max, etc
• Boolean type variables, i.e. with only true/false values, with Is or is to stress this fact, e.g. if dataIsLoaded
• Reuse names for short-life and variables with local scope, such as indexes of loops
• Only use i, j, etc., as indexes for very short loops

3. Miscellaneous

• Add sanity checks to the code, e.g., if something does not work as expected, there should be code to check for this and either issue a warning or an error if there is a problem.
• Do not encode the absolute position of any files within any function: use relative paths, where possible
• Indent the code: really improves readability.
• Fix a maximum line length: break large ones if needed. Ensure that it is clear that the sentence is separated through different lines, e.g.:

function [parameter1, parameter2, parameter3, parameter4] = functionManyParameters...
          (InputParameter1, InputParameter2, InputParameter3, InputParameter3, ...
           InputParameter4, InputParameter5)

• Divide the code in separate functional files whenever it is possible (and logical)

4. Platform independent code

• Use pwd to get the current directory
• Use filesep for paths (e.g., ['myPath' filesep 'myFile.m'])

Documentation and comments

• Make sure the code is fully documented and commented, especially parts of the code that might be difficult to understand for beginner users.
• Header for each file with the following elements:
  • Brief description (easy and short functions) or more detailed explanations (more complicated functions).
  • Description of INPUT and OUTPUT variables
  • Authors, co-authors, contributors (and the contribution of each of them)
  • Date of first fully operative version, and dates of consequent modifications with the corresponding number of version, e.g. v1 - 11/06/2014 / v2 - 12/08/2014
  • Description of modifications in later versions, e.g. v2: the efficiency has been improved by substituting the loops with matrices operations
• Throughout the file:
  • Comment smartly. Not every line, but enough to allow tracking the execution
  • Try to use brief comments.
  • In case you believe a more complicated part requires a more comprehensive explanation, describe What are you doing and How it is done through a more detailed paragraph.
  • If the code is divided in blocks, you can also introduce briefly what is the function of each block beforehand.
  • Format the comments with a whitespace after the % sign. Try to use lowercase letters for comments.

Tests

• Annotate the individual tests extensively for review
• Use assert(computedResult == expectedResult) to logically test the computedResult and the expectedResult (you may also use < or >)
• For testing all entries of a vector, use assert(all(vector1 == vector2))
• Only use equality assert tests for integer values
• Make sure that equality assert tests within a given tolerance, e.g., tol = 1e-9; assert(condition < tol);
• Write loops for testing multiple models and/or solvers
• Try to make your tests compatible with as many solvers as possible
• Test, if possible, your contribution on Linux
• Make sure to limit the output of the function to a minimum - only print the necessary information
• Use verbose or printLevel to switch the verbose mode
• Ensure that the solution of optimization problems is actually a solution (test that the solution vector satisfies the imposed constraints)

Git commit messages

• Use the present tense (“Add feature” not “Added feature”)
• Limit the first line to 72 characters or less
• Reference issues and pull requests liberally
• When only changing documentation, include [documentation] in the commit description
• Consider starting the commit message (not the title of the PR) with an applicable emoji:
  • :bug: when fixing a bug
  • :art: when improving the format/structure of the code
  • :racehorse: when improving performance
  • :memo: when writing docs
  • :fire: when removing code or files
  • :white_check_mark: when adding tests
  • :penguin: when fixing something on Linux
  • :apple: when fixing something on macOS
  • :computer: when fixing something on Windows
  • :green_heart: when fixing the CI build

Issues/enhancements guide

What should I do before opening an issue?

Following these guidelines helps maintainers and the community understand your report , reproduce the behavior , and fix it.

• Get the latest version Check if you can reproduce the problem with the latest version, if the problem happens when you run your script/function with a different version of MATLAB or when using a different computer or different operating system (such as Linux, Windows, or macOS).
• Check The COBRA Toolbox Forum on Google here Has your question already been asked, or has your problem already been reported? If it has, add a comment to the existing issue instead of opening a new one.
• Check the list of currently reported issues here. If you happen to find a related issue, please use that thread or refer in your new issue to an already existing one.
• Check the list of open pull requests here.

If you provide snippets in the issue/pull request, use Markdown code blocks.

How can I report an issue (or enhancement)?

Explain the problem and include additional details to help maintainers reproduce the problem:

• Use a clear and descriptive title.
• Provide information when you last updated The COBRA Toolbox.
• Describe the exact steps that reproduce the problem and that describe the proposed enhancement in as many details as possible. Start by explaining how you started and initialized The COBRA Toolbox.
• Provide specific examples to demonstrate the steps.
• Describe the current behavior you observed after following the steps and point out what exactly is the problem with that behavior.
• Explain what behavior you expected to see instead and why.

Provide details on the nature of the issue:

• If you’re reporting that MATLAB crashed, include a crash report. Indicate the line that you think that made MATLAB crash. As this is an operating system related issue, please make sure to test the same code on a different computer and/or operating system before reporting the issue.
• If the problem is related to performance, include more details, such as CPU information, RAM, computer brand/model. Eventually, mention where you feel potential improvements could be made.
• If the problem wasn’t triggered by a specific action, describe what you were doing before the problem happened.

Include details about your configuration and environment:

• Provide details on your startup.m and your pathdef.m file.
• Which version of MATLAB are you using?
• What operating system are you using?
• Which solvers do you have installed? You can get that table by running initCobraToolbox.

Provide more context by answering these questions:

• Might the problem be related to an external program?
• Did the problem start happening recently?
• If the problem started happening recently, can you reproduce the problem in an older version of The COBRA Toolbox?
• Can you reliably reproduce the issue?
• Does the problem happen for all COBRA models, or only some?