Listeners#

Here we describe the Argilla listeners capabilities:

class argilla.listeners.Metrics(**kwargs)#

Metrics results for a single listener execution.

The metrics object exposes the metrics configured for the listener as property values. For example, if you define a listener including the metric “F1”, the results will be accessible as metrics.F1

class argilla.listeners.RGDatasetListener(dataset, action, metrics=None, query=None, query_params=None, condition=None, query_records=True, interval_in_seconds=30)#

The argilla dataset listener class

Parameters:

  • dataset (str) – The dataset over which listener is created

  • action (Union[Callable[[List[Union[TextClassificationRecord, TokenClassificationRecord, Text2TextRecord, TextGenerationRecord]], RGListenerContext], bool], Callable[[RGListenerContext], bool]]) – The action to execute when the condition is satisfied

  • metrics (Optional[List[str]]) – A list of metrics ids that will be required in condition

  • query (Optional[str]) – The query string to apply

  • query_params (Optional[Dict[str, Any]]) – Defined parameters used dynamically in the provided query

  • condition (Optional[Callable[[Search, Optional[RGListenerContext]], bool]]) – The condition to satisfy to execute the action

  • query_records (bool) – If False, the records won’t be passed as argument to the action. Default: True

  • interval_in_seconds (int) – How often the listener is executed. Default to 30 seconds

property formatted_query: Optional[str]#

Formatted query using defined query params, if any

is_running()#

True if listener is running

start(*action_args, **action_kwargs)#

Start listen to changes in the dataset. Additionally, args and kwargs can be passed to action by using the action_* arguments

If the listener is already started, a ValueError will be raised

stop()#

Stops listener if it’s still running.

If listener is already stopped, a ValueError will be raised

class argilla.listeners.RGListenerContext(listener, search=None, metrics=None, query_params=None)#

The argilla listener execution context. This class keeps the context components related to a listener

Parameters:

  • listener (RGDatasetListener) – The argilla listener instance

  • search (Optional[Search]) – Search results for current execution

  • metrics (Optional[Metrics]) – Metrics results for current execution

  • query_params (Optional[Dict[str, Any]]) – Dynamic parameters used in the listener query

property dataset: str#

Computed property that returns the configured listener dataset name

property query: Optional[str]#

Computed property that returns the configured listener query string

class argilla.listeners.Search(total, query_params=None)#

Search results for a single listener execution

Parameters:

  • total (int) – The total number of records affected by the listener query

  • query_params (Optional[Dict[str, Any]]) – The query parameters applied to the executed search

argilla.listeners.listener(dataset, query=None, metrics=None, condition=None, with_records=True, execution_interval_in_seconds=30, **query_params)#

Configures the decorated function as an argilla listener.

Parameters:

  • dataset (str) – The dataset name.

  • query (Optional[str]) – The query string.

  • metrics (Optional[List[str]]) – Required metrics for listener condition.

  • condition (Optional[Callable[[Search, Optional[RGListenerContext]], bool]]) – Defines condition over search and metrics that launch action when is satisfied.

  • with_records (bool) – Include records as part of action arguments. If False, only the listener context RGListenerContext will be passed. Default: True.

  • execution_interval_in_seconds (int) – Define the execution interval in seconds when listener iteration will be executed.

  • **query_params – Dynamic parameters used in the query. These parameters will be available via the listener context and can be updated for subsequent queries.