QDYN-pylib Development¶
The source code for the qdyn
Python package (“QDYN-pylib”) that accompanies the QDYN Fortran library is organized in the
./qdynpylib
subfolder of the QDYN repository.
While QDYN-pylib is developed as part of QDYN, it is publicly released to the Python Package Index (PyPI) and versioned independently from the Fortran QDYN library. This is so that qdyn
may be used without restrictions in Notebooks or scripts to process data generated by QDYN. Therefore, the ./qdynpylib
folder is self-sufficient for developing and releasing QDYN-pylib, and contains:
The package code in
src/qdyn
(having a src-directory solves problems with testing)Package installation and metadata files
setup.py
,setup.cfg
, andMANIFEST.in
. These determines what goes into the source distribution files on PyPI. Dependencies are defined insetup.py
.The PyPI-documentation files
README.md
andHISTORY.md
. Both of these are combined into the “Projet Description” on PyPI.The test environment configuration file
tox.ini
, wrapped by aMakefile
that runs the tests in thetests
subfolder with pytest.
Testing¶
Testing of QDYN-pylib is done with pytest, just like the testing of the QDYN Fortran code (see the notes on The Testing Framework). There are two different context for the tests:
Self-contained testing, running the tests in the
./qdynpylib/tests
folder under all supported Python versions (using tox)Integration testing within the main QDYN testing framework, running tests in both
./tests
and./qdynpylib/tests
within the virtual environment used for testing the QDYN Fortran code.
Self-contained tests¶
The self-contained tests for QDYN-pylib can be found in the ./qdynpylib/tests
folder. These tests do not rely on having access to an installation of the QDYN Fortran library. They are included in the source distribution uploaded to PyPI, such that any user of the QDYN-pylib package can run the tests to verify their installation of the package.
The tests can be run against all supported Python version with tox, as configured in the tox.ini
file.
To run them, you type e.g.
tox -e py38-test
from inside the ./qdynpylib
folder.
Run
tox -av
to see a list of all tox environments.
Warning
The prerequisites for testing with tox are:
You must have all Python versions installed against which you want to test. Python 3.8 is the main version, so a
python3.8
executable should be in your$PATH
. You must also havepython3.7
andpython3.9
executables in your$PATH
if you want to test against these versionsAt least one of these Python versions must have the tox package installed (
python -m pip install tox
) such that thetox
executable is in your$PATH
The invocation of tox is also wrapped into a Makefile
inside the qdnpylib
folder. While being inside the ./qdynpylib
folder, execute
make test
To run tests for all available Python versions (via tox)
Run simply
make
to see a list of make targets (or, make help
).
Warning
In order to run tox, or use the Makefile
for QDYN-pylib’s self-contained tests, you must be inside the ./qdynpylib
subdirectory of a checkout of the QDYN repository; it will not work if you are in the root of the repository. Moreover, the Makefile
in the repository root is separate from ./qdynpylib/Makefile
and will run the full QDYN Test Suite instead.
QDYN-pylib must pass testing with tox
when making a release; make release
will enforce this automatically.
Testing within the QDYN Test Suite¶
When running
make test
from the root of a checkout of the QDYN repository, pytest will be used to run all the tests of the QDYN projects. This includes the tests of QDYN-pylib in ./qdynpylib/tests
and the tests of the Fortran code in ./tests
. All tests will be run within the ./venv
environment that the configure
script sets up.
The main ./tests
folder may also contain “integration tests” where QDYN-pylib is tested in combination with the Fortran code. Moreover, tests of the Fortran code are encouraged to routinely use QDYN-pylib for generating test data and verifying results, implicitly testing the compatibility of the Fortran and Python code.
If you want to just quickly run the isolated QDYN-pylib tests in the context of the main venv
(and don’t want to bother with tox
as recommended in the previous section), you can run
make test-qdynpylib
Warning
Make sure that you are in the root of the QDYN repository when running make test
or make test-qdynpylib
: The main QDYN Makefile
in the project root is different from the file ./qdynpylib/Makefile
use for self-contained testing, see above.
Coverage¶
When running the full QDYN test suite (make test
in the repository root), coverage data for QDYN-pylib will be collected, and an HTML report will be generated n the ./qdynpylib/htmlcov
. This report shows which line in each Python file has been run during testing. Ideally, the “coverage” (the percentage of lines run) should be above 90%.
When doing self-contained testing (make test
in the ./qdynpylib
subdirectory), a coverage report can also be generated by running make coverage
). Note that this coverage will only include the self-contained tests; coverage will be lower than that reported by the main QDYN test suite. This is fine: we do not need full coverage for the isolated tests. In particular, some parts of the QDYN-pylib such as qdyn.testing
only make sense in the context of the full QDYN repository and cannot be tested in isolation.
Running make clean
either from the repository root for the ./qdynpylib
directory will remove the coverage report.
Code Style¶
All code must be compatible with PEP 8. The line length limit is 79 characters, although exceptions are permissible if this improves readability significantly.
Beyond PEP 8, this project adopts the Black code style. You can run make black-check
or tox -e run-blackcheck
from inside the ./qdynpylib
directory to check adherence to the code style, and make black
or tox -e run-black
to apply it.
Imports within python modules must be sorted according to the isort configuration in ./qdynpylib/setup.cfg
. The command make isort-check
or tox -e run-isortcheck
(run from inside ./qdynpylib
) checks whether all imports are sorted correctly, and make isort
or tox -e run-isort
modifies all Python modules in-place with the proper sorting.
Documentation¶
The documentation of QDYN-pylib is intermixed with the main QDYN documentation (e.g., this very page). Any how-to, example, or topical overview related to QDYN-pylib should be written according to the Writing Documentation Guide.
There are two “self-contained” parts of the QDYN-pylib documentation:
The files
README.md
andHISTORY.md
in the./qdynpylib
directory: These do not show up in the rendered Sphinx documentation, but only on PyPI. They are targeted as a minimal documentation for users who do not have access to the full QDYN, for example someone who just wants to reproduce an analysis for data generated by QDYN. They must not reference any other part of the documentation. The fact that these files are written using Markdown formatting instead of the RestructuredText Syntax used for regular documentation encourages this separation.The API documentation in the form of docstrings in the Python source code, rendered as QDYN-pylib API.
For the API documentation, each function or class must have a docstring; this docstring must be written in the “Google Style” format (as implemented by Sphinx’ napoleon extension).
The __init__
method of a Python class should never have a docstring; it’s arguments are described in the class docstring instead.
Python docstrings use the same RestructuredText Syntax as the Fortran docstrings.
They can include mathematical formulas in LaTeX syntax (using mathjax). Inline math should be enclosed in dollar signs, see the guidelines for writing math in the main QDYN documentation. Use semantic macros such as \ket{\Psi}
instead of \vert\Psi\rangle
.
Use the same conventions for Bibliographic References as in the Fortran API, by putting Local reference lists in module, class or function docstrings when appropriate.
Warning
Within Python docstrings, the “primary domain” is py
, whereas for all other parts of the documnation if it f
(for Fortran). Thus, :mod:`qdyn`
inside a Python docstring is equivalent to py:mod:`qdyn`
and links to qdyn
, wheras :mod:`qdyn`
in any Fortran docstring or RST document is equivalent to f:mod:`qdyn`
and links to qdyn
.
See also References to Fortran objects and References to Python objects.
Use the :f
prefix when referencing Fortran objecs from Python docstrings, and the :py
prefix when referencing Python objects from Fortran docstrings!
Versioning and Releases¶
QDYN-pylib is independently versioned from QDYN as a whole, in order to allow for PyPI releases. The version number is defined in ./qdynpylib/src/qdyn/__init__.py
and must be compatible with PEP 440. It is available through the __version__
attribute of the qdyn
package.
In short, versions number follow the pattern major.minor.patch
, e.g. 0.1.0
for the first release, and 1.0.0
for the first stable release. A major version of 0 indicates that package API has not stabilized and may change in every minor-version release.
Between releases, __version__
on the main branch should either be the version number of the last release, with “+dev” appended (as a “local version identifier”), or the version number of the next planned release, with “-dev” appended (“pre-release identifier” with extra dash). The “+dev” suffix must never be included in a release to PyPI.
The qdyn
package is released to PyPI by running
make release
from inside the ./qdynpylib
folder and following the prompts.
Releases should be tagged with e.g. qdynpylib-v0.1.0
. Running make
release
will do this automatically.