Hopsworks 4.2 Observability

Hopsworks 4.2 Observability 

Hopsworks 4.2 is now generally available. This new release adds support for observability for the Online Feature Store, finishes the series of functional updates related to Transformation Functions and provides an upgrade to the supported java version in the platform along with associated components.

Online Feature Store Observability

With the release of 4.2, Hopsworks has now made significant improvements to the Online Feature Store enabling better observability of data ingestion. Hopsworks users can now more easily track the ingestion of their critical data as it is processed by the platform using API calls. In addition, there is better error reporting when a problem is encountered during ingestion. An example notebook of how this functionality can be used is available here.

Transformation Functions

Hopsworks has introduced a series of functionality updates to enhance how transformation functions are handled within the platform. With the 4.2 release, this series is now complete, making model-independent, model-dependent, and on-demand transformation functions more flexible to handle a variety of use cases. Additionally, Python and Pandas user-defined functions (UDFs) are now supported across all types of transformation functions. For more details check out the detailed documentation here.

Java Upgrade

Hopsworks now supports Java 17 as part of the 4.2 upgrade. This allows better memory management of a number of different scheduled jobs in the Hopsworks platform and also allows a number of high performance internal components of Hopsworks to be upgraded. For example, one such component is the Java client for reading and writing of parquet files to/from Apache Hudi.

Packages Update

With the Hopsworks 4.2 release, a number of different packages have been updated including vLLM to 0.7.1, JupyterLab to 4.2.5 and a number of other related packages to allow the latest LLMs to be used by Hopsworks users.

Bug Fixes

Featurestore

FSTORE-1605 - Add global path field in the storage connector

FSTORE-1640 - Delta streamer supports repeatable reads

FSTORE-1661 - Arrowflight data queries can run on nodes with no client certificate registered

Ancillary Services

HWORKS-2501 - Configurable DNS port when patching CoreDNS ConfigMap for Consul

HWORKS-1957 - Allow RonDB helm to include explicit database names besides ignore configuration

HWORKS-1902 - Handle helm upgrades in remote shuffle service