Unit Testing
It is important to have unit-tests to verify that your CLI is behaving correctly. For unit-testing, we will be using the defacto-standard python unit-testing library, pytest. This section demonstrates some common scenarios you may encounter when unit-testing your CLI app.
Lets make a small application that checks PyPI if a library name is available:
# pypi_checker.py
import sys
import urllib.error
import urllib.request
import cyclopts
def _check_pypi_name_available(name):
try:
urllib.request.urlopen(f"https://pypi.org/pypi/{name}/json")
except urllib.error.HTTPError as e:
if e.code == 404:
return True # Package does not exist (name is available)
return False # Package exists (name is not available)
app = cyclopts.App(
config=[
cyclopts.config.Env("PYPI_CHECKER_"),
cyclopts.config.Json("config.json"),
],
)
@app.default
def pypi_checker(name: str, *, silent: bool = False):
"""Check if a package name is available on PyPI.
Exit code 0 on success; non-zero otherwise.
Parameters
----------
name: str
Name of the package to check.
silent: bool
Do not print anything to stdout.
"""
is_available = _check_pypi_name_available(name)
if not silent:
if is_available:
print(f"{name} is available.")
else:
print(f"{name} is not available.")
sys.exit(not is_available)
if __name__ == "__main__":
app()
Running the app from the console:
$ python pypi_checker.py --help
Usage: pypi_checker COMMAND [ARGS] [OPTIONS]
Check if a package name is available on PyPI.
Exit code 0 on success; non-zero otherwise.
╭─ Commands ────────────────────────────────────────────────────────────────────╮
│ --help -h Display this message and exit. │
│ --version Display application version. │
╰───────────────────────────────────────────────────────────────────────────────╯
╭─ Parameters ──────────────────────────────────────────────────────────────────╮
│ * NAME --name Name of the package to check. [required] │
│ --silent --no-silent Do not print anything to stdout. [default: False] │
╰───────────────────────────────────────────────────────────────────────────────╯
$ python pypi_checker.py cyclopts
cyclopts is not available.
$ python pypi_checker.py cyclopts --silent
$ echo $? # Check the exit code of the previous command.
1
$ python pypi_checker.py the-next-big-project
the-next-big-project is available.
$ echo $? # Check the exit code of the previous command.
0
We will slowly introduce unit-testing concepts and build up a fairly comprehensive set of unit-tests for this application.
Mocking
First off, it's good code-hygiene to separate "business logic" from "user interface."
In this example, that means putting all the actual logic of determining whether or not a package name is available into the _check_pypi_name_available
function, and putting all of the CLI logic (like printing to stdout
and exit-codes) in the Cyclopts-decorated function pypi_checker
.
This makes it easier to unit-test the app because it allows us to mock out portions of our app, allowing us to isolate our CLI unit-tests to just the CLI components.
We can use pytest-mock to simplify mocking _check_pypi_name_available
. Let's define a fixture that declares this mock.
# test.py
import pytest
from pypi_checker import app
@pytest.fixture
def mock_check_pypi_name_available(mocker):
return mocker.patch("pypi_checker._check_pypi_name_available")
Unit tests that use this fixture can define it's return value, as well as check the arguments it was called with. This will be demonstrated in the next section.
Exit Codes
Our app directly calls sys.exit()
.
Internal to python, this causes the SystemExit
exception to be raised.
We can catch this with the pytest.raises()
context manager, and check the resulting error-code.
def test_unavailable_name(mock_check_pypi_name_available):
mock_check_pypi_name_available.return_value = False
with pytest.raises(SystemExit) as e:
app("foo") # Invoke our app, passing in package-name "foo"
mock_check_pypi_name_available.assert_called_once_with("foo") # assert that our mock was called.
assert e.value.code != 0 # assert the exit code is non-zero (i.e. not successful)
We can then run pytest on this file:
$ pytest test.py
============================== test session starts ==============================
platform darwin -- Python 3.13.0, pytest-8.3.4, pluggy-1.5.0
rootdir: /cyclopts-demo
configfile: pyproject.toml
plugins: cov-6.0.0, anyio-4.8.0, mock-3.14.0
collected 1 item
test.py . [100%]
=============================== 1 passed in 0.05s ===============================
Note
Alternatively, we could have avoided using sys.exit()
within our commands, and have our commands instead return an integer error-code.
# pypi_checker.py
@app.default
def pypi_checker(name: str, *, silent: bool = False):
...
return not is_available
if __name__ == "__main__":
sys.exit(app())
With this setup, our unit-test would just have to check:
# test.py
assert app("foo") != 0
Checking stdout
We also want to make sure that our message is displayed to the user.
The built-in capsys fixture gives us access to our application's stdout
.
We can use this to confirm our app prints the correct statement.
# test.py - continued from "Mocking"
def test_unavailable_name(capsys, mock_check_pypi_name_available):
mock_check_pypi_name_available.return_value = False
with pytest.raises(SystemExit) as e:
app("foo") # Invoke our app, passing in package-name "foo"
mock_check_pypi_name_available.assert_called_once_with("foo") # assert that our mock was called.
assert e.value.code != 0 # assert the exit code is non-zero (i.e. not successful)
assert capsys.readouterr().out == "foo is not available.\n"
Environment Variables
Because we configured our App
with cyclopts.config.Env
, we can pass arguments into our application via environment variables.
The pytest monkeypatch fixture allows us to modify environment variables within the context of a unit-test.
In this test, we only want to test if our environment variable is being passed in correctly.
We will use App.parse_args()
, which performs all the parsing, but doesn't actually invoke the command.
# test.py
def test_name_env_var(monkeypatch):
from pypi_checker import pypi_checker
monkeypatch.setenv("PYPI_CHECKER_NAME", "foo")
command, bound, _ = app.parse_args([]) # An empty list - no CLI arguments passed in.
assert command == pypi_checker
assert bound.arguments['name'] == "foo"
Warning
A common mistake is accidentally calling app()
or app.parse_args()
with the intent of providing no arguments.
Calling these methods with no arguments will read from sys.argv
, the same as in a typical application.
This is rarely the intention in a unit-test, and Cyclopts will produce a warning.
For example, this code in a unit test:
app() # Wrong: will produce a warning
Will generate this warning:
=============================== warnings summary ================================
test.py::test_no_args
/my_project/test.py:64: UserWarning: Cyclopts application invoked without tokens
under unit-test framework "pytest". Did you mean "app([])"?
app()
The proper way to specify no CLI arguments is to provide an empty string or list:
app([])
File Config
To explicitly test that configurations from the Cyclopts configuration system are loading properly, we can create a configuration file in a temporary directory and change our current-working-directory (cwd) to that temporary directory. The pytest built-in tmp_path
fixture gives us a temporary directory, and the monkeypatch
fixture allows us to change the cwd. We have to change the cwd because typically configuration files are discovered relative to the directory where the CLI was invoked. If your CLI searches other locations (such as the home directory), you will need to modify this example appropriately.
# test.py
import json
@pytest.fixture(autouse=True)
def chdir_to_tmp_path(tmp_path, monkeypatch):
"Automatically change current directory to tmp_path"
monkeypatch.chdir(tmp_path)
@pytest.fixture
def config_path(tmp_path):
"Path to JSON configuration file in tmp_path"
return tmp_path / "config.json" # same name that was provided to cyclopts.config.Json
def test_config(config_path):
with config_path.open("w") as f:
json.dump({"name": "bar"}, f)
command, bound, _ = app.parse_args([]) # An empty list - no CLI arguments passed in.
assert command == pypi_checker
assert bound.arguments['name'] == "foo"
Help Page
Cyclopts uses Rich to pretty-print messages to the console. Rich interprets the console environment, and can change how it displays text depending on the terminal's capabilities. For unit testing, we will explicitly set a lot of these parameters in a pytest fixture to make it easier to compare against known good values:
@pytest.fixture
def console():
from rich.console import Console
return Console(width=70, force_terminal=True, highlight=False, color_system=None, legacy_windows=False)
Since the help-page is just printed to stdout
, we will be using the capsys fixture again.
from textwrap import dedent
def test_help_page(capsys, console):
app("--help", console=console)
actual = capsys.readouterr().out
assert actual == dedent(
"""\
Usage: pypi-checker COMMAND [ARGS] [OPTIONS]
Check if a package name is available on PyPI.
Exit code 0 on success; non-zero otherwise.
╭─ Commands ─────────────────────────────────────────────────────────╮
│ --help -h Display this message and exit. │
│ --version Display application version. │
╰────────────────────────────────────────────────────────────────────╯
╭─ Parameters ───────────────────────────────────────────────────────╮
│ * NAME --name Name of the package to check. [required] │
│ --silent --no-silent Do not print anything to stdout. │
│ [default: False] │
╰────────────────────────────────────────────────────────────────────╯
"""
)
The textwrap.dedent()
function allows us to have our expected-help-string nicely indented within our code.
Alternatively, we could have used the rich.console.Console.capture()
context manager to directly capture the rich.console.Console
output.
Note
Unit-testing the help-page is probably overkill for most projects (and may get in the way more often than it helps!).