2026-09-14 00:06:13.148 | INFO     | pipecat:<module>:54 - ᓚᘏᗢ Pipecat 0.0.0.dev0 (Python 3.13.15 (main, Aug 12 2026, 00:00:00) [GCC 11.5.0 20240719 (Red Hat 11.5.0-14)]) ᓚᘏᗢ
2026-09-14 00:15:30.545 | INFO | [run_id=None] | storage.py:94 | Initializing storage backend: MINIO (value: minio, ENABLE_AWS_S3=False)
2026-09-14 00:15:30.847 | INFO | [run_id=None] | storage.py:39 | Initializing minio storage at localhost:9000 (public: http://localhost:9000) with bucket 'voice-audio'
2026-09-14 00:20:33.846 | WARNING | [run_id=None] | logging_config.py:48 | Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPConnectionPool(host='localhost', port=9000): Read timed out. (read timeout=300)")': /voice-audio?location=
2026-09-14 00:25:35.847 | WARNING | [run_id=None] | logging_config.py:48 | Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPConnectionPool(host='localhost', port=9000): Read timed out. (read timeout=300)")': /voice-audio?location=
2026-09-14 01:20:17,229 | ERROR | arq.worker | 125.90s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 711, in checkout
    rec = pool._do_get()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/impl.py", line 175, in _do_get
    return self._create_connection()
           ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 388, in _create_connection
    return _ConnectionRecord(self)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 673, in __init__
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 01:46:18,626 | WARNING | arq.worker | 666.00s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 01:46:21,924 | WARNING | arq.worker | 584.21s ! cron:sweep_inactive_text_chat_sessions max retries 1 exceeded
2026-09-14 02:15:00,129 | ERROR | arq.worker | 1286.60s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 711, in checkout
    rec = pool._do_get()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/impl.py", line 175, in _do_get
    return self._create_connection()
           ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 388, in _create_connection
    return _ConnectionRecord(self)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 673, in __init__
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:15:11,829 | ERROR | arq.worker | 1352.40s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 820, in _async_ping
    await tr.start()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 146, in start
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:15:12,627 | ERROR | arq.worker | 2323.50s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<4 lines>...
        conn,
        ^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
        statement, params or {}, execution_options=execution_options
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2358, in _handle_dbapi_exception
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        cursor, str_statement, effective_parameters, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
    self._adapt_connection.await_(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._prepare_and_execute(operation, parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 521, in _prepare_and_execute
    prepared_stmt, attributes = await adapt_connection._prepare(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        operation, self._invalidate_schema_cache_asof
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 768, in _prepare
    prepared_stmt = await self._connection.prepare(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        operation, name=self._prepared_statement_name_func()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 635, in prepare
    return await self._prepare(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 653, in _prepare
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 447, in _get_statement
    types, intro_stmt = await self._introspect_types(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        types_with_missing_codecs, timeout)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 514, in _introspect_types
    result = await self.__execute(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2024, in _do_execute
    result = await executor(stmt, None)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 206, in bind_execute
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:15:15,730 | ERROR | arq.worker | 2241.60s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<4 lines>...
        conn,
        ^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
        statement, params or {}, execution_options=execution_options
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2358, in _handle_dbapi_exception
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        cursor, str_statement, effective_parameters, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
    self._adapt_connection.await_(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._prepare_and_execute(operation, parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 521, in _prepare_and_execute
    prepared_stmt, attributes = await adapt_connection._prepare(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        operation, self._invalidate_schema_cache_asof
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 768, in _prepare
    prepared_stmt = await self._connection.prepare(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        operation, name=self._prepared_statement_name_func()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 635, in prepare
    return await self._prepare(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 653, in _prepare
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 447, in _get_statement
    types, intro_stmt = await self._introspect_types(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        types_with_missing_codecs, timeout)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 497, in _introspect_types
    cfgrow, _ = await self.__execute(
                ^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2004, in _do_execute
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 432, in _get_statement
    statement = await self._protocol.prepare(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "asyncpg/protocol/protocol.pyx", line 165, in prepare
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:15:36,924 | WARNING | arq.worker | 1917.10s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 02:15:40,326 | WARNING | arq.worker | 1854.00s ! cron:sweep_inactive_text_chat_sessions max retries 1 exceeded
2026-09-14 02:16:00,728 | ERROR | arq.worker | 1626.30s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 822, in _async_ping
    await self._connection.fetchrow(";")
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 748, in fetchrow
    data = await self._execute(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1864, in _execute
    result, _ = await self.__execute(
                ^^^^^^^^^^^^^^^^^^^^^
    ...<7 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2004, in _do_execute
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 490, in _get_statement
    await self._cleanup_stmts()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1641, in _cleanup_stmts
    await self._protocol.close_statement(stmt, protocol.NO_TIMEOUT)
  File "asyncpg/protocol/protocol.pyx", line 594, in close_statement
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:18:17,624 | ERROR | arq.worker | 118.00s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 934, in __connect_addr
    await connected
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:18:18,926 | ERROR | arq.worker |  90.70s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:18:28,029 | ERROR | arq.worker | 1819.90s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 822, in _async_ping
    await self._connection.fetchrow(";")
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 748, in fetchrow
    data = await self._execute(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1864, in _execute
    result, _ = await self.__execute(
                ^^^^^^^^^^^^^^^^^^^^^
    ...<7 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2004, in _do_execute
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 490, in _get_statement
    await self._cleanup_stmts()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1641, in _cleanup_stmts
    await self._protocol.close_statement(stmt, protocol.NO_TIMEOUT)
  File "asyncpg/protocol/protocol.pyx", line 594, in close_statement
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:20:48,328 | ERROR | arq.worker |  71.00s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 934, in __connect_addr
    await connected
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:22:33,327 | ERROR | arq.worker | 151.20s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 934, in __connect_addr
    await connected
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:29:51,830 | ERROR | arq.worker | 158.21s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 810, in _create_ssl_connection
    do_ssl_upgrade = await pr.on_data
                     ^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:32:46,823 | ERROR | arq.worker |  88.10s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:33:36,933 | WARNING | arq.worker | 1117.51s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 02:35:08,124 | WARNING | arq.worker | 1208.60s ! cron:sweep_inactive_text_chat_sessions max retries 1 exceeded
2026-09-14 02:41:27,807 | ERROR | arq.worker | 212.28s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:55:40,409 | ERROR | arq.worker | 844.69s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:55:42,412 | ERROR | arq.worker | 2388.09s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<4 lines>...
        conn,
        ^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
        statement, params or {}, execution_options=execution_options
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2358, in _handle_dbapi_exception
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        cursor, str_statement, effective_parameters, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
    self._adapt_connection.await_(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._prepare_and_execute(operation, parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 521, in _prepare_and_execute
    prepared_stmt, attributes = await adapt_connection._prepare(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        operation, self._invalidate_schema_cache_asof
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 768, in _prepare
    prepared_stmt = await self._connection.prepare(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        operation, name=self._prepared_statement_name_func()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 635, in prepare
    return await self._prepare(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 653, in _prepare
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 447, in _get_statement
    types, intro_stmt = await self._introspect_types(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        types_with_missing_codecs, timeout)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 514, in _introspect_types
    result = await self.__execute(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2024, in _do_execute
    result = await executor(stmt, None)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 206, in bind_execute
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 02:55:42,904 | ERROR | arq.worker | 2391.18s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<4 lines>...
        conn,
        ^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
        statement, params or {}, execution_options=execution_options
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2358, in _handle_dbapi_exception
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        cursor, str_statement, effective_parameters, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
    self._adapt_connection.await_(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._prepare_and_execute(operation, parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 521, in _prepare_and_execute
    prepared_stmt, attributes = await adapt_connection._prepare(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        operation, self._invalidate_schema_cache_asof
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 768, in _prepare
    prepared_stmt = await self._connection.prepare(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        operation, name=self._prepared_statement_name_func()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 635, in prepare
    return await self._prepare(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 653, in _prepare
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 447, in _get_statement
    types, intro_stmt = await self._introspect_types(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        types_with_missing_codecs, timeout)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 514, in _introspect_types
    result = await self.__execute(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2024, in _do_execute
    result = await executor(stmt, None)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 206, in bind_execute
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 03:18:05,514 | WARNING | arq.worker | 465.40s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 03:22:21,818 | WARNING | arq.worker | 592.90s ! cron:sweep_inactive_text_chat_sessions max retries 1 exceeded
2026-09-14 03:25:37,216 | ERROR | arq.worker | 788.90s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 824, in _async_ping
    await tr.rollback()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 227, in rollback
    await self.__rollback()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 206, in __rollback
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 03:26:54,018 | ERROR | arq.worker | 742.71s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 824, in _async_ping
    await tr.rollback()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 227, in rollback
    await self.__rollback()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 206, in __rollback
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 03:28:48,722 | WARNING | arq.worker | 768.90s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 03:30:45,618 | WARNING | arq.worker | 875.20s ! cron:sweep_inactive_text_chat_sessions max retries 1 exceeded
2026-09-14 03:30:51,224 | WARNING | arq.worker | 573.90s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 03:32:35,523 | ERROR | arq.worker |  62.49s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 810, in _create_ssl_connection
    do_ssl_upgrade = await pr.on_data
                     ^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 03:39:50,818 | ERROR | arq.worker | 382.60s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 03:40:32,123 | ERROR | arq.worker | 1191.11s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 824, in _async_ping
    await tr.rollback()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 227, in rollback
    await self.__rollback()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 206, in __rollback
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 03:41:54,821 | ERROR | arq.worker | 932.90s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<4 lines>...
        conn,
        ^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
        statement, params or {}, execution_options=execution_options
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2358, in _handle_dbapi_exception
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        cursor, str_statement, effective_parameters, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
    self._adapt_connection.await_(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._prepare_and_execute(operation, parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 515, in _prepare_and_execute
    await adapt_connection._start_transaction()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 843, in _start_transaction
    await self._transaction.start()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 146, in start
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 03:45:10,921 | ERROR | arq.worker | 119.20s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 03:51:23,421 | ERROR | arq.worker | 1897.70s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 822, in _async_ping
    await self._connection.fetchrow(";")
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 748, in fetchrow
    data = await self._execute(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1864, in _execute
    result, _ = await self.__execute(
                ^^^^^^^^^^^^^^^^^^^^^
    ...<7 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2004, in _do_execute
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 490, in _get_statement
    await self._cleanup_stmts()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1641, in _cleanup_stmts
    await self._protocol.close_statement(stmt, protocol.NO_TIMEOUT)
  File "asyncpg/protocol/protocol.pyx", line 594, in close_statement
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 03:55:12,926 | ERROR | arq.worker | 211.41s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 03:58:59,820 | ERROR | arq.worker | 103.10s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 04:23:32,019 | WARNING | arq.worker | 491.19s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 04:31:25,922 | WARNING | arq.worker | 663.80s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 04:32:23,120 | WARNING | arq.worker | 582.50s ! cron:sweep_inactive_text_chat_sessions max retries 1 exceeded
2026-09-14 04:39:24,122 | ERROR | arq.worker | 186.10s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1147, in create_connection
    sock = await self._connect_sock(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
        exceptions, addrinfo, laddr_infos)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1046, in _connect_sock
    await self.sock_connect(sock, address)
  File "/usr/lib64/python3.13/asyncio/selector_events.py", line 643, in sock_connect
    return await fut
           ^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 711, in checkout
    rec = pool._do_get()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/impl.py", line 175, in _do_get
    return self._create_connection()
           ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 388, in _create_connection
    return _ConnectionRecord(self)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 673, in __init__
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 04:39:40,025 | ERROR | arq.worker | 255.00s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 810, in _create_ssl_connection
    do_ssl_upgrade = await pr.on_data
                     ^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 711, in checkout
    rec = pool._do_get()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/impl.py", line 175, in _do_get
    return self._create_connection()
           ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 388, in _create_connection
    return _ConnectionRecord(self)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 673, in __init__
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 04:40:41,424 | ERROR | arq.worker | 1494.00s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<4 lines>...
        conn,
        ^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
        statement, params or {}, execution_options=execution_options
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2358, in _handle_dbapi_exception
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        cursor, str_statement, effective_parameters, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
    self._adapt_connection.await_(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._prepare_and_execute(operation, parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 545, in _prepare_and_execute
    self._rows = deque(await prepared_stmt.fetch(*parameters))
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/prepared_stmt.py", line 176, in fetch
    data = await self.__bind_execute(args, 0, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/prepared_stmt.py", line 267, in __bind_execute
    data, status, _ = await self.__do_execute(
                      ^^^^^^^^^^^^^^^^^^^^^^^^
        lambda protocol: protocol.bind_execute(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            self._state, args, '', limit, True, timeout))
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/prepared_stmt.py", line 256, in __do_execute
    return await executor(protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 206, in bind_execute
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 04:49:31,920 | WARNING | arq.worker | 1238.91s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 04:50:43,219 | WARNING | arq.worker | 1317.20s ! cron:sweep_inactive_text_chat_sessions max retries 1 exceeded
2026-09-14 05:10:10,221 | ERROR | arq.worker | 1076.10s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 05:10:11,526 | ERROR | arq.worker | 2693.30s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 822, in _async_ping
    await self._connection.fetchrow(";")
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 748, in fetchrow
    data = await self._execute(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1864, in _execute
    result, _ = await self.__execute(
                ^^^^^^^^^^^^^^^^^^^^^
    ...<7 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2004, in _do_execute
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 432, in _get_statement
    statement = await self._protocol.prepare(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "asyncpg/protocol/protocol.pyx", line 165, in prepare
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 05:10:18,432 | ERROR | arq.worker | 2879.71s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 822, in _async_ping
    await self._connection.fetchrow(";")
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 748, in fetchrow
    data = await self._execute(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1864, in _execute
    result, _ = await self.__execute(
                ^^^^^^^^^^^^^^^^^^^^^
    ...<7 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2004, in _do_execute
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 490, in _get_statement
    await self._cleanup_stmts()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1641, in _cleanup_stmts
    await self._protocol.close_statement(stmt, protocol.NO_TIMEOUT)
  File "asyncpg/protocol/protocol.pyx", line 594, in close_statement
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 05:10:18,830 | ERROR | arq.worker | 2213.91s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<4 lines>...
        conn,
        ^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
        statement, params or {}, execution_options=execution_options
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2358, in _handle_dbapi_exception
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        cursor, str_statement, effective_parameters, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
    self._adapt_connection.await_(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._prepare_and_execute(operation, parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 515, in _prepare_and_execute
    await adapt_connection._start_transaction()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 843, in _start_transaction
    await self._transaction.start()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 146, in start
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 05:10:20,520 | ERROR | arq.worker | 2259.30s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<4 lines>...
        conn,
        ^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
        statement, params or {}, execution_options=execution_options
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2358, in _handle_dbapi_exception
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        cursor, str_statement, effective_parameters, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
    self._adapt_connection.await_(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._prepare_and_execute(operation, parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 515, in _prepare_and_execute
    await adapt_connection._start_transaction()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 843, in _start_transaction
    await self._transaction.start()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 146, in start
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 05:48:00,423 | WARNING | arq.worker | 929.60s ! cron:sweep_inactive_text_chat_sessions max retries 1 exceeded
2026-09-14 05:49:41,521 | WARNING | arq.worker | 1003.70s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 05:53:35,621 | WARNING | arq.worker | 944.90s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 05:55:15,421 | WARNING | arq.worker | 1044.20s ! cron:sweep_inactive_text_chat_sessions max retries 1 exceeded
2026-09-14 06:23:31,331 | ERROR | arq.worker | 1182.81s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 711, in checkout
    rec = pool._do_get()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/impl.py", line 175, in _do_get
    return self._create_connection()
           ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 388, in _create_connection
    return _ConnectionRecord(self)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 673, in __init__
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 06:23:32,929 | ERROR | arq.worker | 1266.01s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 820, in _async_ping
    await tr.start()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 146, in start
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 06:23:33,025 | ERROR | arq.worker | 2747.20s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 822, in _async_ping
    await self._connection.fetchrow(";")
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 748, in fetchrow
    data = await self._execute(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1864, in _execute
    result, _ = await self.__execute(
                ^^^^^^^^^^^^^^^^^^^^^
    ...<7 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2004, in _do_execute
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 490, in _get_statement
    await self._cleanup_stmts()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1641, in _cleanup_stmts
    await self._protocol.close_statement(stmt, protocol.NO_TIMEOUT)
  File "asyncpg/protocol/protocol.pyx", line 594, in close_statement
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 06:23:40,826 | ERROR | arq.worker | 2316.70s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 820, in _async_ping
    await tr.start()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 146, in start
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 06:23:42,326 | ERROR | arq.worker | 2899.60s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 203, in greenlet_spawn
    result = context.switch(value)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 822, in _async_ping
    await self._connection.fetchrow(";")
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 748, in fetchrow
    data = await self._execute(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1864, in _execute
    result, _ = await self.__execute(
                ^^^^^^^^^^^^^^^^^^^^^
    ...<7 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2004, in _do_execute
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 490, in _get_statement
    await self._cleanup_stmts()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1641, in _cleanup_stmts
    await self._protocol.close_statement(stmt, protocol.NO_TIMEOUT)
  File "asyncpg/protocol/protocol.pyx", line 594, in close_statement
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 06:23:42,925 | ERROR | arq.worker | 2527.40s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 203, in greenlet_spawn
    result = context.switch(value)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 822, in _async_ping
    await self._connection.fetchrow(";")
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 748, in fetchrow
    data = await self._execute(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1864, in _execute
    result, _ = await self.__execute(
                ^^^^^^^^^^^^^^^^^^^^^
    ...<7 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2004, in _do_execute
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 432, in _get_statement
    statement = await self._protocol.prepare(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "asyncpg/protocol/protocol.pyx", line 165, in prepare
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 06:52:17,126 | WARNING | arq.worker | 694.60s ! cron:sweep_inactive_text_chat_sessions max retries 1 exceeded
2026-09-14 06:55:33,526 | ERROR | arq.worker | 850.00s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 824, in _async_ping
    await tr.rollback()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 227, in rollback
    await self.__rollback()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 206, in __rollback
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 06:55:36,527 | WARNING | arq.worker | 604.61s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 06:55:37,121 | ERROR | arq.worker | 367.80s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 203, in greenlet_spawn
    result = context.switch(value)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 822, in _async_ping
    await self._connection.fetchrow(";")
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 748, in fetchrow
    data = await self._execute(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1864, in _execute
    result, _ = await self.__execute(
                ^^^^^^^^^^^^^^^^^^^^^
    ...<7 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2024, in _do_execute
    result = await executor(stmt, None)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 206, in bind_execute
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 07:20:39,425 | WARNING | arq.worker | 338.00s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 07:20:45,022 | ERROR | arq.worker | 310.40s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<4 lines>...
        conn,
        ^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
        statement, params or {}, execution_options=execution_options
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2358, in _handle_dbapi_exception
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        cursor, str_statement, effective_parameters, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
    self._adapt_connection.await_(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._prepare_and_execute(operation, parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 515, in _prepare_and_execute
    await adapt_connection._start_transaction()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 843, in _start_transaction
    await self._transaction.start()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 146, in start
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 07:31:36,024 | ERROR | arq.worker |  66.60s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 08:01:21,019 | WARNING | arq.worker | 363.70s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 08:03:52,023 | WARNING | arq.worker | 475.30s ! cron:sweep_inactive_text_chat_sessions max retries 1 exceeded
2026-09-14 08:07:43,920 | WARNING | arq.worker | 448.69s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 08:12:14,726 | ERROR | arq.worker | 216.40s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 711, in checkout
    rec = pool._do_get()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/impl.py", line 175, in _do_get
    return self._create_connection()
           ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 388, in _create_connection
    return _ConnectionRecord(self)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 673, in __init__
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 08:12:17,622 | ERROR | arq.worker | 443.60s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 824, in _async_ping
    await tr.rollback()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 227, in rollback
    await self.__rollback()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 206, in __rollback
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 08:12:17,821 | ERROR | arq.worker | 1012.00s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<4 lines>...
        conn,
        ^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
        statement, params or {}, execution_options=execution_options
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2358, in _handle_dbapi_exception
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        cursor, str_statement, effective_parameters, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
    self._adapt_connection.await_(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._prepare_and_execute(operation, parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 521, in _prepare_and_execute
    prepared_stmt, attributes = await adapt_connection._prepare(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        operation, self._invalidate_schema_cache_asof
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 768, in _prepare
    prepared_stmt = await self._connection.prepare(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        operation, name=self._prepared_statement_name_func()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 635, in prepare
    return await self._prepare(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 653, in _prepare
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 447, in _get_statement
    types, intro_stmt = await self._introspect_types(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        types_with_missing_codecs, timeout)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 497, in _introspect_types
    cfgrow, _ = await self.__execute(
                ^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2004, in _do_execute
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 432, in _get_statement
    statement = await self._protocol.prepare(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "asyncpg/protocol/protocol.pyx", line 165, in prepare
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 08:12:26,223 | ERROR | arq.worker | 880.80s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<4 lines>...
        conn,
        ^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
        statement, params or {}, execution_options=execution_options
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2358, in _handle_dbapi_exception
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        cursor, str_statement, effective_parameters, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
    self._adapt_connection.await_(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._prepare_and_execute(operation, parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 515, in _prepare_and_execute
    await adapt_connection._start_transaction()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 843, in _start_transaction
    await self._transaction.start()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 146, in start
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 08:12:35,425 | ERROR | arq.worker | 622.50s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 824, in _async_ping
    await tr.rollback()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 227, in rollback
    await self.__rollback()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 206, in __rollback
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 08:21:26,921 | ERROR | arq.worker |  65.10s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 934, in __connect_addr
    await connected
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 08:31:44,522 | WARNING | arq.worker | 346.50s ! cron:sweep_inactive_text_chat_sessions max retries 1 exceeded
2026-09-14 08:33:58,821 | ERROR | arq.worker | 126.90s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 08:34:06,824 | ERROR | arq.worker | 461.40s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<4 lines>...
        conn,
        ^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
        statement, params or {}, execution_options=execution_options
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2358, in _handle_dbapi_exception
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        cursor, str_statement, effective_parameters, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
    self._adapt_connection.await_(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._prepare_and_execute(operation, parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 545, in _prepare_and_execute
    self._rows = deque(await prepared_stmt.fetch(*parameters))
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/prepared_stmt.py", line 176, in fetch
    data = await self.__bind_execute(args, 0, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/prepared_stmt.py", line 267, in __bind_execute
    data, status, _ = await self.__do_execute(
                      ^^^^^^^^^^^^^^^^^^^^^^^^
        lambda protocol: protocol.bind_execute(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            self._state, args, '', limit, True, timeout))
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/prepared_stmt.py", line 256, in __do_execute
    return await executor(protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 206, in bind_execute
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 08:47:24,522 | ERROR | arq.worker | 414.40s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 802, in _create_ssl_connection
    tr, pr = await loop.create_connection(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        host, port)
        ^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1123, in create_connection
    infos = await self._ensure_resolved(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        (host, port), family=family,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        type=socket.SOCK_STREAM, proto=proto, flags=flags, loop=self)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 1519, in _ensure_resolved
    return await loop.getaddrinfo(host, port, family=family, type=type,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                  proto=proto, flags=flags)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/base_events.py", line 940, in getaddrinfo
    return await self.run_in_executor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        None, getaddr_func, host, port, family, type, proto, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 08:53:41,125 | ERROR | arq.worker |  71.60s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2421, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<20 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 1049, in _connect
    conn = await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 886, in _connect_addr
    return await __connect_addr(params, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 931, in __connect_addr
    tr, pr = await connector
             ^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connect_utils.py", line 810, in _create_ssl_connection
    do_ssl_upgrade = await pr.on_data
                     ^^^^^^^^^^^^^^^^
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1264, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 714, in checkout
    dbapi_connection = rec.get_connection()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 836, in get_connection
    self.__connect()
    ~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 899, in __connect
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 895, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/create.py", line 661, in connect
    return dialect.connect(*cargs, **cparams)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 629, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)  # type: ignore[no-any-return]  # NOQA: E501
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 964, in connect
    await_only(creator_fn(*arg, **kw)),
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2420, in connect
    async with compat.timeout(timeout):
               ~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 09:38:29,624 | WARNING | arq.worker | 467.60s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 09:39:18,128 | WARNING | arq.worker | 493.50s ! cron:sweep_inactive_text_chat_sessions max retries 1 exceeded
2026-09-14 09:39:49,823 | ERROR | arq.worker | 536.20s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<4 lines>...
        conn,
        ^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
        statement, params or {}, execution_options=execution_options
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2358, in _handle_dbapi_exception
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        cursor, str_statement, effective_parameters, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
    self._adapt_connection.await_(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._prepare_and_execute(operation, parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 545, in _prepare_and_execute
    self._rows = deque(await prepared_stmt.fetch(*parameters))
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/prepared_stmt.py", line 176, in fetch
    data = await self.__bind_execute(args, 0, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/prepared_stmt.py", line 267, in __bind_execute
    data, status, _ = await self.__do_execute(
                      ^^^^^^^^^^^^^^^^^^^^^^^^
        lambda protocol: protocol.bind_execute(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            self._state, args, '', limit, True, timeout))
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/prepared_stmt.py", line 256, in __do_execute
    return await executor(protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 206, in bind_execute
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 09:39:51,126 | ERROR | arq.worker | 402.81s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 820, in _async_ping
    await tr.start()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/transaction.py", line 146, in start
    await self._connection.execute(query)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 349, in execute
    result = await self._protocol.query(query, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 375, in query
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 10:09:23,323 | WARNING | arq.worker | 550.00s ! cron:sweep_webhook_deliveries max retries 1 exceeded
2026-09-14 10:09:33,822 | ERROR | arq.worker | 514.30s ! cron:sweep_webhook_deliveries failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/webhook_delivery.py", line 399, in sweep_webhook_deliveries
    due = await db_client.get_due_webhook_deliveries(
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        now=datetime.now(UTC), limit=page_size, after_id=after_id
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/webhook_delivery_client.py", line 233, in get_due_webhook_deliveries
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2251, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self,
        ^^^^^
    ...<4 lines>...
        conn,
        ^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement
    result = conn.execute(
        statement, params or {}, execution_options=execution_options
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute
    return meth(
        self,
        distilled_parameters,
        execution_options or NO_OPTIONS,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection
    return connection._execute_clauseelement(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self, distilled_params, execution_options
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement
    ret = self._execute_context(
        dialect,
    ...<8 lines>...
        cache_hit=cache_hit,
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
           ~~~~~~~~~~~~~~~~~~~~~~~~~^
        dialect, context, statement, parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        e, str_statement, effective_parameters, cursor, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 2358, in _handle_dbapi_exception
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        cursor, str_statement, effective_parameters, context
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute
    cursor.execute(statement, parameters)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 580, in execute
    self._adapt_connection.await_(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._prepare_and_execute(operation, parameters)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 521, in _prepare_and_execute
    prepared_stmt, attributes = await adapt_connection._prepare(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        operation, self._invalidate_schema_cache_asof
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 768, in _prepare
    prepared_stmt = await self._connection.prepare(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        operation, name=self._prepared_statement_name_func()
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 635, in prepare
    return await self._prepare(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 653, in _prepare
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 432, in _get_statement
    statement = await self._protocol.prepare(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "asyncpg/protocol/protocol.pyx", line 165, in prepare
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError
2026-09-14 10:09:34,822 | WARNING | arq.worker | 514.69s ! cron:sweep_inactive_text_chat_sessions max retries 1 exceeded
2026-09-14 10:09:34,924 | ERROR | arq.worker | 454.50s ! cron:sweep_inactive_text_chat_sessions failed, TimeoutError: 
Traceback (most recent call last):
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 507, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/tasks/text_chat_inactivity.py", line 41, in sweep_inactive_text_chat_sessions
    inactive_sessions = await db_client.get_inactive_workflow_run_text_sessions(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/api/db/workflow_run_text_session_client.py", line 123, in get_inactive_workflow_run_text_sessions
    result = await session.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
    ...<15 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/ext/asyncio/session.py", line 463, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
    ...<6 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 203, in greenlet_spawn
    result = context.switch(value)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2365, in execute
    return self._execute_internal(
           ~~~~~~~~~~~~~~~~~~~~~~^
        statement,
        ^^^^^^^^^^
    ...<4 lines>...
        _add_event=_add_event,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2241, in _execute_internal
    conn = self._connection_for_bind(bind)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 2110, in _connection_for_bind
    return trans._connection_for_bind(engine, execution_options)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 2, in _connection_for_bind
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/state_changes.py", line 137, in _go
    ret_value = fn(self, *arg, **kw)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/orm/session.py", line 1189, in _connection_for_bind
    conn = bind.connect()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3277, in connect
    return self._connection_cls(self)
           ~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 143, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/base.py", line 3301, in raw_connection
    return self.pool.connect()
           ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 447, in connect
    return _ConnectionFairy._checkout(self)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1363, in _checkout
    with util.safe_reraise():
         ~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/langhelpers.py", line 224, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/pool/base.py", line 1301, in _checkout
    result = pool._dialect._do_ping_w_event(
        fairy.dbapi_connection
    )
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/engine/default.py", line 728, in _do_ping_w_event
    return self.do_ping(dbapi_connection)
           ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 1169, in do_ping
    dbapi_connection.ping()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 811, in ping
    _ = self.await_(self._async_ping())
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 132, in await_only
    return current.parent.switch(awaitable)  # type: ignore[no-any-return,attr-defined] # noqa: E501
           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 822, in _async_ping
    await self._connection.fetchrow(";")
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 748, in fetchrow
    data = await self._execute(
           ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1864, in _execute
    result, _ = await self.__execute(
                ^^^^^^^^^^^^^^^^^^^^^
    ...<7 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1961, in __execute
    result, stmt = await self._do_execute(
                   ^^^^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 2004, in _do_execute
    stmt = await self._get_statement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 490, in _get_statement
    await self._cleanup_stmts()
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/asyncpg/connection.py", line 1641, in _cleanup_stmts
    await self._protocol.close_statement(stmt, protocol.NO_TIMEOUT)
  File "asyncpg/protocol/protocol.pyx", line 594, in close_statement
asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bigbrainscrm/voice.bigbrainscrm.com/venv/lib64/python3.13/site-packages/arq/worker.py", line 594, in run_job
    result = await asyncio.wait_for(task, timeout_s)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/tasks.py", line 506, in wait_for
    async with timeouts.timeout(timeout):
               ~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/asyncio/timeouts.py", line 117, in __aexit__
    raise TimeoutError from exc_val
TimeoutError