How to Configure JupyterHub Idle Culler on JupyterHub

[Linux Hint]

JupyterHub is a multi-user Jupyter notebook server. Since the JupyterHub server is used by many users at the same time, a lot of system resources will be used. To optimize the usage of system resources for JupyterHub, you can keep only the active user sessions of JupyterHub and open and close the idle ones. The JupyterHub idle culler service does just that. The JupyterHub idle culler services close the idle user sessions to make sure that more active user sessions can use the freed-up system resources.

In this article, we will show you how to install the JupyterHub idle culler on the JupyterHub virtual environment and configure JupyterHub to use it.

NOTE: If you don’t have JupyterHub installed on your computer, you can read one of the articles depending on the Linux distribution you’re using:

1. How to Install the Latest Version of JupyterHub on Ubuntu 22.04 LTS/ Debian 12/Linux Mint 21

2. How to Install the Latest Version of JupyterHub on Fedora 38+/RHEL 9/Rocky Linux 9

Topic of Contents:

1. Installing JupyterHub Idle Culler
2. Configuring JupyterHub Idle Culler
3. Restarting the JupyterHub Service
4. Testing If JupyterHub IDLE Culler Is Working Correctly
5. Conclusion
6. References

Installing JupyterHub Idle Culler

If you followed our JupyterHub Installation Guide to install JupyterHub on your favorite Linux distributions (Debian-based and RPM-based), you can install the JupyterHub idle culler in the JupyterHub virtual environment with the following command:

$ sudo /opt/jupyterhub/bin/python3 -m pip install jupyterhub-idle-culler

The JupyterHub idle culler should now be installed in the JupyterHub virtual environment.

Configuring JupyterHub Idle Culler

To configure the JupyterHub idle culler, open the JupyterHub configuration file with the nano text editor as follows:

$ sudo nano /opt/jupyterhub/etc/jupyterhub/jupyterhub_config.py

Add the following lines of codes in the “jupyterhub_config.py” file:

# initialize Jupyter Hub empty roles and services
c.JupyterHub.load_roles = list()
c.JupyterHub.services = list()

# Configure Jupyter Hub idle culler service
idle_culler_role = {
    'name': 'jupyterhub-idle-culler-role',
    'scopes': [
        'list:users',
        'read:users:activity',
        'read:servers',
        'delete:servers',
        'admin:users'
    ],
    'services': ['jupyterhub-idle-culler-service']
}

import sys
SESSION_TIMEOUT = 3600
idle_culler_service = {
    'name': 'jupyterhub-idle-culler-service',
    'command': [
        sys.executable,
        '-m', 'jupyterhub_idle_culler',
        f"--timeout={SESSION_TIMEOUT}"
    ]
}

c.JupyterHub.load_roles.append(idle_culler_role)
c.JupyterHub.services.append(idle_culler_service)

Here, SESSION_TIMEOUT is the number of seconds after a JupyterHub user session becomes idle and the IDLE culler stops/closes the session. We set it to 3600 seconds or 1 hour. You can change it depending on your requirements.

Once you’re done, press <Ctrl> + X followed by “Y” and <Enter> to save the “jupyterhub_config.py” file.

Restarting the JupyterHub Service

For the JupyterHub configuration changes to take effect, restart the JupyterHub “systemd” service with the following command:

$ sudo systemctl restart jupyterhub.service

If the JupyterHub configuration file is error-free, the JupyterHub “systemd” service should be running^[1] and the JupyterHub IDLE Culler service should also be running^[2] as you can see in the following screenshot:

$ sudo systemctl status jupyterhub.service

Testing iIf JupyterHub IDLE Culler Is Working Correctly

To verify whether the JupyterHub idle culler is stopping the idle use sessions, log in to JupyterHub as any user and don’t refresh the page for an hour (as we configured JupyterHub to stop the user sessions that are idle for 3600 seconds/60 minutes/1 hour). As you can see, the user session is running.

After an hour, refresh the page and you should see that the user session is stopped automatically. It means that the JupyterHub idle culler is working as expected.

Conclusion

In this article, we showed you how to install the JupyterHub idle culler service on the JupyterHub virtual environment. We also showed you how to configure the JupyterHub idle culler so that the JupyterHub idle user sessions are stopped automatically to free the system resources for other JupyterHub active users.

References:

• GitHub – jupyterhub/jupyterhub-idle-culler: JupyterHub service to cull idle servers and users
• JupyterHub – ArchWiki

View the full article