Hopsworks 4.4 External Data Services

‍Hopsworks 4.4 is now generally available. This new release introduces the possibility of key Hopsworks data services to run outside of a kubernetes cluster in conjunction with all other services running within a customer’s kubernetes cluster. This increases the flexibility of how Hopsworks customers can deploy, maintain and operate the Hopsworks MLOps platform within their infrastructure whether that be on the cloud or on-prem. In addition, with the 4.4 release, Hopsworks has introduced a python API for the management of Alerts within the Hopsworks platform.

External Data Services

Enterprises across a number of verticals including finance, insurance, entertainment and retail have a variety of policies related to the handling of sensitive data using their infrastructure whether that be on the cloud or on-prem. To address this, the Hopsworks 4.4 release allows different data services to be run either inside or outside of a Hopsworks kubernetes cluster. The data services that can run in this fashion with the 4.4 release include: RonDB, OpenSearch, HopsFS and Prometheus.

Given that these external data services can now run outside of a kubernetes cluster, the management of these external data services is achieved using Ansible. By using Ansible, all the standard operation aspects of these external data services are supported (i.e., started, stopped, take backups, etc).

Python Alerts API

With the Hopsworks 4.4 release, it is now possible to interact with all the functionality provided by the Alerts management system in the Hopsworks platform using the python API. More details of how this can be achieved is outlined here.

Bug Fixes

Featurestore

FSTORE-1745 - Add read only role for S3 storage connector

FSTORE-1769 - Improve data preview performance for dynamic queries

FSTORE-1770 - Uninstalling Python Libraries fails with a server error if a failed custom command execution is persistent

Ancillary Services

HWORKS-2255 - Upgrade doesn't work after the introduction of filebeat container for streaming audit logs

HWORKS-2210 - LDAP principal containing space can not be set

HWORKS-2230 - Optional alternative registry/url for Hopsworks base images