❓ FAQ¶

Here is a list of questions we have either been asked by users or potential pitfalls we hope to help users avoid:

Q: When to use UniDep?¶

A: UniDep is particularly useful for setting up full development environments that require both Python and non-Python dependencies (e.g., CUDA, compilers, etc.) with a single command.

In fields like research, data science, robotics, AI, and ML projects, it is common to work from a locally cloned Git repository.

Setting up a full development environment can be a pain, especially if you need to install non Python dependencies like compilers, low-level numerical libraries, or CUDA (luckily Conda has all of them). Typically, instructions are different for each OS and their corresponding package managers (apt, brew, yum, winget, etc.).

With UniDep, you can specify all your Pip and Conda dependencies in a single file. To get set up on a new machine, you just need to install Conda (we recommend micromamba) and run pip install unidep; unidep install-all -e in your project directory, to install all dependencies and local packages in editable mode in the current Conda environment.

For fully reproducible environments, you can run unidep conda-lock to generate a conda-lock.yml file. Then, run conda env create -f conda-lock.yml -n myenv to create a new Conda environment with all the third-party dependencies. Finally, run unidep install-all -e --no-dependencies to install all your local packages in editable mode.

For those who prefer not to use Conda, you can simply run pip install -e . on a project using UniDep. You’ll need to install the non-Python dependencies yourself, but you’ll have a list of them in the requirements.yaml file.

In summary, use UniDep if you:

Prefer installing packages with conda but still want your package to be pip installable.
Are tired of synchronizing your Pip requirements (requirements.txt) and Conda requirements (environment.yaml).
Want a low-effort, comprehensive development environment setup.

Q: Just show me a full example!¶

A: Check out the example folder.

Q: Uses of UniDep in the wild?¶

A: UniDep really shines when used in a monorepo with multiple dependent projects, however, since these are typically private, we cannot share them.

However, an example of a single package that is public is home-assistant-streamdeck-yaml. This is a Python package that allows to interact with Home Assistant from an Elgato Stream Deck connected via USB to e.g., a Raspberry Pi. It requires a couple of system dependencies (e.g., libusb and hidapi), which are typically installed with apt or brew. The README.md shows different installation instructions on Linux, MacOS, and Windows for non-Conda installs, however, with UniDep, we can just use unidep install . on all platforms. It is fully configured via pyproject.toml. The 2 Dockerfiles show 2 different ways of using UniDep:

Dockerfile.locked: Installing conda-lock.yml (generated with unidep conda-lock) and then pip install . the local package.
Dockerfile.latest: Using unidep install . to install all dependencies, first with conda, then pip, then the local package.

Q: How is this different from conda/mamba/pip?¶

A: UniDep uses pip and conda under the hood to install dependencies, but it is not a replacement for them. UniDep will print the commands it runs, so you can see exactly what it is doing.

Q: I found a project using unidep, now what?¶

A: You can install it like any other Python package using pip install. However, to take full advantage of UniDep’s functionality, clone the repository and run unidep install-all -e in the project directory. This installs all dependencies in editable mode in the current Conda environment.

Q: How to handle local dependencies that do not use UniDep?¶

A: You can use the local_dependencies field in the requirements.yaml or pyproject.toml file to specify local dependencies. However, if a local dependency is not managed by UniDep, it will skip installing its dependencies!

To include all its dependencies, either convert the package to use UniDep (🏆), or maintain a separate requirements.yaml file, e.g., for a package called foo create, foo-requirements.yaml:

dependencies:
  # List the dependencies of foo here
  - numpy
  - scipy
  - matplotlib
  - bar
local_dependencies:
  - ./path/to/foo  # This is the path to the package

Then, in the requirements.yaml or pyproject.toml file of the package that uses foo, list foo-requirements.yaml as a local dependency:

local_dependencies:
  - ./path/to/foo-requirements.yaml

Q: Can’t Conda already do this?¶

A: Not quite. Conda can indeed install both Conda and Pip dependencies via an environment.yaml file, however, it does not work the other way around. Pip cannot install the pip dependencies from an environment.yaml file. This means, that if you want your package to be installable with pip install -e . and support Conda, you need to maintain two separate files: environment.yaml and requirements.txt (or specify these dependencies in pyproject.toml or setup.py).

Q: What is the difference between `conda-lock` and `unidep conda-lock`?¶

A: conda-lock is a standalone tool that creates a conda-lock.yml file from a environment.yaml file. On the other hand, unidep conda-lock is a command within the UniDep tool that also generates a conda-lock.yml file (leveraging conda-lock), but it does so from one or more requirements.yaml or pyproject.toml files. When managing multiple dependent projects (e.g., in a monorepo), a unique feature of unidep conda-lock is its ability to create consistent individual conda-lock.yml files for each requirements.yaml or pyproject.toml file, ensuring consistency with a global conda-lock.yml file. This feature is not available in the standalone conda-lock tool.

Q: What is the difference between `hatch-conda` / `pdm-conda` and `unidep`?¶

A: hatch-conda is a plugin for hatch that integrates Conda environments into hatch. A key difference is that hatch-conda keeps Conda and Pip dependencies separate, choosing to install packages with either Conda or Pip. This results in Conda being a hard requirement, for example, if numba is specified for Conda, it cannot be installed with Pip despite its availability on PyPI.

In contrast, UniDep does not require Conda. Without Conda, it can still install any dependency that is available on PyPI (e.g., numba is both Conda and Pip installable). However, without Conda, UniDep will not install dependencies exclusive to Conda. These Conda-specific dependencies can often be installed through alternative package managers like apt, brew, yum, or by building them from source.

Another key difference is that hatch-conda is managing Hatch environments whereas unidep can install Pip dependencies in the current Python environment (venv, Conda, Hatch, etc.), however, to optimally use UniDep, we recommend using Conda environments to additionally install non-Python dependencies.

Similar to hatch-conda, unidep also integrates with Hatchling, but it works in a slightly different way.

A: pdm-conda is a plugin for pdm designed to facilitate the use of Conda environments in conjunction with pdm. Like hatch-conda, pdm-conda opts to install packages either with Conda or Pip. It is closely integrated with pdm, primarily enabling the inclusion of Conda packages in pdm’s lock file (pdm.lock). However, pdm-conda lacks extensive cross-platform support. For instance, when adding a package like Numba using pdm-conda, it gets locked to the current platform (e.g., osx-arm64) without the flexibility to specify compatibility for other platforms such as linux64. In contrast, UniDep allows for cross-platform compatibility, enabling the user to specify dependencies for multiple platforms. UniDep currently does not support pdm, but it does support Hatchling and Setuptools.

UniDep stands out from both pdm-conda and hatch-conda with its additional functionalities, particularly beneficial for monorepos and projects spanning multiple operating systems. For instance:

Conda Lock Files: Create conda-lock.yml files for all packages with consistent sub-lock files per package.
CLI tools: Provides tools like unidep install-all -e which will install multiple local projects (e.g., in monorepo) and all its dependencies first with Conda, then remaining ones with Pip, and finally the local dependencies in editable mode with Pip.
Conda Environment Files: Can create standard Conda environment.yaml files by combining the dependencies from many requirements.yaml or pyproject.toml files.
Platform-Specific Dependencies: Allows specifying dependencies for certain platforms (e.g., linux64, osx-arm64), enhancing cross-platform compatibility.