Hopsworks 4.7: On-Demand Transformations, Feature Sharing...

Hopsworks 4.7 is now generally available. This release delivers on-demand transformations for online-offline feature parity, cross-project feature sharing and federated OpenSearch search, alongside major component upgrades including Python 3.11, Spark 3.5.5, Kafka 3.9.0, and continued RonDB 24.10 refinements.

On-Demand Transformations

On-demand transformations allow features that depend on request-time parameters to be computed dynamically during online inference, without introducing online-offline skew. A single transformation function can produce one or more on-demand features, and functions are executed automatically both when inserting data into a feature group and when retrieving feature vectors at inference time. Real-time parameters are passed via the request_parameter argument in get_feature_vector() and get_feature_vectors(), and on-demand features integrate seamlessly with feature views alongside model-dependent transformations. In 4.7, transformation functions can now also directly read features selected as helper columns without requiring them to be manually passed as request parameters, reducing boilerplate in inference pipelines.

Cross-Project Feature Sharing

Hopsworks 4.7 introduces the ability to share feature groups and individual features between projects. Teams can now expose selected feature groups or fine-grained sets of features from one project for use in the queries and feature views of another project, enabling reuse of curated, governed feature sets across organisational boundaries without duplicating data pipelines. This is accompanied by search improvements that allow users to find shared assets by tags and keywords, and support for mandatory tags that enforce governance policies across feature groups.

Federated OpenSearch

Hopsworks 4.7 adds federated OpenSearch support, enabling cross-cluster search across Hopsworks deployments. This allows a single query to span feature metadata, model assets, and lineage information stored across multiple clusters, supporting larger enterprise deployments and multi-region architectures where data must remain distributed while still being uniformly discoverable.

Software Upgrades

All Hopsworks base environments now standardise on Python 3.11, the Spark runtime and Spark history server have been upgraded to 3.5.5, the Kafka cluster managed by Hopsworks is upgraded to Kafka 3.9.0 running on Strimzi 0.45 and the vLLM serving image was upgraded to 0.14.0 with updated PyTorch and OpenAI-compatible serving images.

Bug Fixes

Featurestore

FSTORE-1906: Online schema validation improved; schema validation is disabled when writing feature logs to avoid false rejections.

FSTORE-1900: Inserting a DataFrame with duplicate primary keys into a Delta feature group led to row duplication.

FSTORE-1867: Complex features in a spine group were incorrectly decoded during training data generation

Ancillary Services

HWORKS-2501: Set an upper version bound for the fsspec dependency to prevent incompatible upgrades.

HWORKS-2469: Creating an OAuth client from Helm values failed with a SQL error when group_claim was set.

HWORKS-2508: CPU resource type was incorrectly typed as integer instead of string in Helm schemas.

HWORKS-2309: Execution ID label change in Hopsworks required a corresponding update to the Logstash pipeline.