# 📝 `requirements.yaml` and `pyproject.toml` structure

`unidep` allows either using a
1. `requirements.yaml` file with a specific format (similar but _**not**_ the same as a Conda `environment.yaml` file) or
2. `pyproject.toml` file with a `[tool.unidep]` section.

Both files contain the following keys:

- **name** (Optional): For documentation, not used in the output.
- **channels**: List of conda channels for packages, such as `conda-forge`.
- **dependencies**: Mix of Conda and Pip packages.
- **local_dependencies** (Optional): List of paths to other `requirements.yaml` or `pyproject.toml` files to include.
- **optional_dependencies** (Optional): Dictionary with lists of optional dependencies.
- **platforms** (Optional): List of platforms that are supported (used in `conda-lock`).

Whether you use a `requirements.yaml` or `pyproject.toml` file, the same information can be specified in either.
Choose the format that works best for your project.

## Example

### Example `requirements.yaml`

Example of a `requirements.yaml` file:

```yaml
name: example_environment
channels:
  - conda-forge
dependencies:
  - numpy                   # same name on conda and pip
  - conda: python-graphviz  # When names differ between Conda and Pip
    pip: graphviz
  - pip: slurm-usage >=1.1.0,<2  # pip-only
  - conda: mumps                 # conda-only
  # Use platform selectors
  - conda: cuda-toolkit =11.8    # [linux64]
local_dependencies:
  - ../other-project-using-unidep     # include other projects that use unidep
  - ../common-requirements.yaml       # include other requirements.yaml files
  - ../project-not-managed-by-unidep  # 🚨 Skips its dependencies!
optional_dependencies:
  test:
    - pytest
  full:
    - ../other-local-dep[test]  # include its optional 'test' dependencies
platforms:  # (Optional) specify platforms that are supported (used in conda-lock)
  - linux-64
  - osx-arm64
```

```{important}
`unidep` can process this during `pip install` and create a Conda installable `environment.yaml` or `conda-lock.yml` file, and more!
```

```{note}
For a more in-depth example containing multiple installable projects, see the [`example`](https://github.com/basnijholt/unidep/tree/main/example/) directory.
```

### Example `pyproject.toml`

***Alternatively***, one can fully configure the dependencies in the `pyproject.toml` file in the `[tool.unidep]` section:

```toml
[tool.unidep]
channels = ["conda-forge"]
dependencies = [
    "numpy",                                         # same name on conda and pip
    { conda = "python-graphviz", pip = "graphviz" }, # When names differ between Conda and Pip
    { pip = "slurm-usage >=1.1.0,<2" },              # pip-only
    { conda = "mumps" },                             # conda-only
    { conda = "cuda-toolkit =11.8:linux64" }         # Use platform selectors by appending `:linux64`
]
local_dependencies = [
    "../other-project-using-unidep",    # include other projects that use unidep
    "../common-requirements.yaml",      # include other requirements.yaml files
    "../project-not-managed-by-unidep"  # 🚨 Skips its dependencies!
]
optional_dependencies = {
    test = ["pytest"],
    full = ["../other-local-dep[test]"]  # include its optional 'test' dependencies
}
platforms = [ # (Optional) specify platforms that are supported (used in conda-lock)
    "linux-64",
    "osx-arm64"
]
```

This data structure is *identical* to the `requirements.yaml` format, with the exception of the `name` field and the [platform selectors](requirementsyaml-and-pyprojecttoml-structure.md#platform-selectors).
In the `requirements.yaml` file, one can use e.g., `# [linux64]`, which in the `pyproject.toml` file is `:linux64` at the end of the package name.

See [Build System Integration](build-system-integration.md#build-system-integration) for more information on how to set up `unidep` with different build systems (Setuptools or Hatchling).

```{important}
In these docs, we often mention the `requirements.yaml` format for simplicity, but the same information can be specified in `pyproject.toml` as well.
Everything that is possible in `requirements.yaml` is also possible in `pyproject.toml`!
```

## Key Points

- Standard names (e.g., `- numpy`) are assumed to be the same for Conda and Pip.
- Use a dictionary with `conda: <package>` *and* `pip: <package>` to specify different names across platforms.
- Use `pip:` to specify packages that are only available through Pip.
- Use `conda:` to specify packages that are only available through Conda.
- Use `# [selector]` (YAML only) or `package:selector` to specify platform-specific dependencies.
- Use `local_dependencies:` to include other `requirements.yaml` or `pyproject.toml` files and merge them into one. Also allows projects that are not managed by `unidep` to be included, but be aware that this skips their dependencies! Can specify PyPI alternatives for monorepo setups (see [PyPI Alternatives for Local Dependencies](build-system-integration.md#pypi-alternatives-for-local-dependencies)).
- Use `optional_dependencies:` to specify optional dependencies. Can be installed like `unidep install ".[test]"` or `pip install ".[test]"`.
- Use `platforms:` to specify the platforms that are supported. If omitted, all platforms are assumed to be supported.

> *We use the YAML notation here, but the same information can be specified in `pyproject.toml` as well.*

## Supported Version Pinnings

UniDep supports a range of version pinning operators (the same as Conda):

- **Standard Version Constraints**: Specify exact versions or ranges with standard operators like `=`, `>`, `<`, `>=`, `<=`.
  - Example: `=1.0.0`, `>1.0.0, <2.0.0`.

- **Version Exclusions**: Exclude specific versions using `!=`.
  - Example: `!=1.5.0`.

- **Redundant Pinning Resolution**: Automatically resolves redundant version specifications.
  - Example: `>1.0.0, >0.5.0` simplifies to `>1.0.0`.

- **Contradictory Version Detection**: Errors are raised for contradictory pinnings to maintain dependency integrity. See the [Conflict Resolution](requirementsyaml-and-pyprojecttoml-structure.md#conflict-resolution) section for more information.
  - Example: Specifying `>2.0.0, <1.5.0` triggers a `VersionConflictError`.

- **Invalid Pinning Detection**: Detects and raises errors for unrecognized or improperly formatted version specifications.

- **Conda Build Pinning**: UniDep also supports Conda's build pinning, allowing you to specify builds in your pinning patterns.
  - Example: Conda supports pinning builds like `qsimcirq * cuda*` or `vtk * *egl*`.
  - **Limitation**: While UniDep allows such build pinning, it requires that there be a single pin per package. UniDep cannot resolve conflicts where multiple build pinnings are specified for the same package.
    - Example: UniDep can handle `qsimcirq * cuda*`, but it cannot resolve a scenario with both `qsimcirq * cuda*` and `qsimcirq * cpu*`.

- **Other Special Cases**: In addition to Conda build pins, UniDep supports all special pinning formats, such as VCS (Version Control System) URLs or local file paths. This includes formats like `package @ git+https://git/repo/here` or `package @ file:///path/to/package`. However, UniDep has a limitation: it can handle only one special pin per package. These special pins can be combined with an unpinned version specification, but not with multiple special pin formats for the same package.
  - Example: UniDep can manage dependencies specified as `package @ git+https://git/repo/here` and `package` in the same `requirements.yaml`. However, it cannot resolve scenarios where both `package @ git+https://git/repo/here` and `package @ file:///path/to/package` are specified for the same package.

```{caution}
**Pinning Validation and Combination**: UniDep actively validates and/or combines pinnings only when **multiple different pinnings** are specified for the same package.
This means if your `requirements.yaml` files include multiple pinnings for a single package, UniDep will attempt to resolve them into a single, coherent specification.
However, if the pinnings are contradictory or incompatible, UniDep will raise an error to alert you of the conflict.
```

## Conflict Resolution

`unidep` features a conflict resolution mechanism to manage version conflicts and platform-specific dependencies in `requirements.yaml` or `pyproject.toml` files.

### How It Works

- **Version Pinning Priority**: `unidep` gives priority to version-pinned packages when the same package is specified multiple times. For instance, if both `foo` and `foo <1` are listed, `foo <1` is selected due to its specific version pin.

- **Platform-Specific Version Pinning**: `unidep` resolves platform-specific dependency conflicts by preferring the version with the narrowest platform scope. For instance, given `foo <3 # [linux64]` and `foo >1`, it installs `foo >1,<3` exclusively on Linux-64 and `foo >1` on all other platforms.

- **Intractable Conflicts**: When conflicts are irreconcilable (e.g., `foo >1` vs. `foo <1`), `unidep` raises an exception.

## Platform Selectors

This tool supports a range of platform selectors that allow for specific handling of dependencies based on the user's operating system and architecture. This feature is particularly useful for managing conditional dependencies in diverse environments.

### Supported Selectors

The following selectors are supported:

- `linux`: For all Linux-based systems.
- `linux64`: Specifically for 64-bit Linux systems.
- `aarch64`: For Linux systems on ARM64 architectures.
- `ppc64le`: For Linux on PowerPC 64-bit Little Endian architectures.
- `osx`: For all macOS systems.
- `osx64`: Specifically for 64-bit macOS systems.
- `arm64`: For macOS systems on ARM64 architectures (Apple Silicon).
- `macos`: An alternative to `osx` for macOS systems.
- `unix`: A general selector for all UNIX-like systems (includes Linux and macOS).
- `win`: For all Windows systems.
- `win64`: Specifically for 64-bit Windows systems.

### Usage

Selectors are used in `requirements.yaml` files to conditionally include dependencies based on the platform:

```yaml
dependencies:
  - some-package >=1  # [unix]
  - another-package   # [win]
  - special-package   # [osx64]
  - pip: cirq         # [macos win]
    conda: cirq       # [linux]
```

Or when using `pyproject.toml` instead of `requirements.yaml`:

```toml
[tool.unidep]
dependencies = [
    "some-package >=1:unix",
    "another-package:win",
    "special-package:osx64",
    { pip = "cirq:macos win", conda = "cirq:linux" },
]
```

In this example:

- `some-package` is included only in UNIX-like environments (Linux and macOS).
- `another-package` is specific to Windows.
- `special-package` is included only for 64-bit macOS systems.
- `cirq` is managed by `pip` on macOS and Windows, and by `conda` on Linux. This demonstrates how you can specify different package managers for the same package based on the platform.

Note that the `package-name:unix` syntax can also be used in the `requirements.yaml` file, but the `package-name # [unix]` syntax is not supported in `pyproject.toml`.

### Implementation

`unidep` parses these selectors and filters dependencies according to the platform where it's being installed.
It is also used for creating environment and lock files that are portable across different platforms, ensuring that each environment has the appropriate dependencies installed.

## `[project.dependencies]` in `pyproject.toml` handling

The `project_dependency_handling` option in `[tool.unidep]` (in `pyproject.toml`) controls how dependencies listed in the standard `[project.dependencies]` section of `pyproject.toml` are handled when processed by `unidep`.

**Modes:**

- **`ignore`** (default): Dependencies in `[project.dependencies]` are ignored by `unidep`.
- **`same-name`**: Dependencies in `[project.dependencies]` are treated as dependencies with the same name for both Conda and Pip. They will be added to the `dependencies` list in `[tool.unidep]` under the assumption that the package name is the same for both package managers.
- **`pip-only`**: Dependencies in `[project.dependencies]` are treated as pip-only dependencies. They will be added to the `dependencies` list in `[tool.unidep]` under the `pip` key.

**Example `pyproject.toml`:**

```toml
[build-system]
requires = ["hatchling", "unidep"]
build-backend = "hatchling.build"

[project]
name = "my-project"
version = "0.1.0"
dependencies = [  # These will be handled according to the `project_dependency_handling` option
  "requests",
  "pandas",
]

[tool.unidep]
project_dependency_handling = "same-name"  # Or "pip-only", "ignore"
dependencies = [
    {conda = "python-graphviz", pip = "graphivz"},
]
```

**Notes:**

- The `project_dependency_handling` option only affects how dependencies from `[project.dependencies]` are processed. Dependencies directly listed under `[tool.unidep.dependencies]` are handled as before.
- This feature is helpful for projects that are already using the standard `[project.dependencies]` field and want to integrate `unidep` without duplicating their dependency list.
- The `project_dependency_handling` feature is _*only available*_ when using `pyproject.toml` files. It is not supported in `requirements.yaml` files.