Developer documentation#
Here we provide some guides for the development of Argilla.
Requirements#
ElasticSearch or OpenSearch#
Argilla supports ElasticSearch and OpenSearch as his main search engine. One of the two is required to correctly run Argilla in your development environment.
Note
Argilla supports ElasticSearch versions 8.8, 8.5, 8.0, and 7.17 and OpenSearch versions 1.3 and 2.3.
For more information please visit our Setup and Installation section.
SQLite#
By default Argilla will use SQLite to store information about users, workspaces, etc. Non additional configuration is required to start using SQLite.
By default the database file will be created at ~/.argilla/argilla.db
, this can be configured setting different values for ARGILLA_DATABASE_URL
and ARGILLA_HOME_PATH
environment variables.
Development setup#
Forking and config your Argilla Git repository#
To set up your system for Argilla development, you first of all have to fork our repository and clone the fork to your computer:
git clone https://github.com/[your-github-username]/argilla.git
cd argilla
To keep your forkโs main/develop branch up to date with our repo you should add it as an upstream remote branch:
git remote add upstream https://github.com/argilla-io/argilla.git
Install dependencies#
To be able to use Argilla via either the Python client or the Python Command Line Interface (CLI), youโll need to
install argilla
and its extra dependencies, if applicable. To do so, you can either install it via pip
or setup
a Conda environment with all the required dependencies.
Via pip
#
If youโre using pip
, you can either install just argilla
or argilla
with any combination or all the extras available.
Weโll install it in editable mode using the -e/--editable
flag
in the pip
command to avoid having to re-install it on every code modification, but if youโre not planning to modify the code, you
can just omit the -e/--editable
flag.
pip install -e .
Or installing just the server
extra:
pip install -e ".[server]"
Or installing all the extras, that are also required to run the tests via pytest
to make sure that the implemented features or the bug fixes work as expected, and that the unit/integration tests are passing.
pip install -e ".[server,listeners,postgresql,integrations,tests]"
Via conda
#
If youโre using conda
, you can go ahead and create a new conda development environment, and then, activate it:
conda env create -f environment_dev.yml
conda activate argilla
In the new Conda environment, Argilla will already be installed in editable mode
with all the server dependencies. But if youโre willing to install any other dependency you can do so via pip
to install your own, or just
see the available extras besides the server
extras, which are: listeners
, postgresql
, and tests
; all those installable as pip install -e ".[<EXTRA_NAME>]"
.
Code formatting tools#
To keep a consistent code format, we use pre-commit hooks. So on, you first need to install
pre-commit
if not installed already, via pip as follows:
pip install pre-commit
Then, you can proceed with the pre-commit
hooks installation by simply running:
pre-commit install
Building Frontend static files#
Build the static UI files in case you want to work on the UI:
bash scripts/build_frontend.sh
Running database migrations#
Run database migrations executing the following task:
python -m argilla database migrate
The default SQLite database will be created at ~/.argilla/argilla.db
. This can be changed setting different values for ARGILLA_DATABASE_URL
and ARGILLA_HOME_PATH
environment variables.
Generating a database migration#
If you have updated one of the SQLAlchemy ORM classes in src/argilla/server/models/models.py
or added a new one, then you will need to generate a new migration script that you can later apply to the database. To generate a new migration script execute the following command:
alembic -c src/argilla/alembic.ini revision --autogenerate -m "descriptive message here"
alembic
will automatically detect the changes made and generate a new migration script in the src/argilla/server/alembic/versions
directory.
Note
After generating a new migration script, you will need to apply the migrations as described in running database migrations.
Recreating the database#
Occasionally, it may be necessary to recreate the database from scratch to ensure a clean state in your development environment. For instance, to run the Argilla test suite or troubleshoot issues that could be related to database inconsistencies.
First you need to delete the Argilla database with the following command:
rm ~/.argilla/argilla.db
After deleting the database, you will need to run the database migrate task:
python -m argilla database migrate
By following these steps, youโll have a fresh and clean database to work with.
Creating your first user#
At least one user is required to interact with Argila API and web UI. You can create easily create your user executing the following task:
python -m argilla users create
This task will ask you for the required information to create your user, including username
, password
and so on.
Running Argilla server#
Finally to run the web app now simply execute:
python -m argilla server
Congrats, you are ready to take Argilla to the next level ๐
Building the documentation#
To build the documentation, make sure you set up your system by installing the required dependencies:
pip install -r docs/_source/requirements.txt
Then use sphinx-autobuild
to continuously deploy the webpage using the following command:
sphinx-autobuild docs/_source docs/_build/html
This will create a _build/html
folder that is served at http://127.0.0.1:8000. Also, it starts watching for changes in the docs/source
directory. When a change is detected in docs/source
, the documentation is rebuilt and any open browser windows are reloaded automatically. KeyboardInterrupt (ctrl+c) will stop the server.
Troubleshooting#
If you get warnings while building documentation then you can handle them this way: If they are toctree
warnings then they can be ignored and If they are import errors then they can be resolved by installing autodoc
and argilla
from docs/_source/requirements.txt