Hopsworks Feature Store provides an open ecosystem that connects to the largest number of data storage, data pipelines, and data science platforms. With support to any Python or Spark environment, there is no vendor lock-in.
Connect to your data warehouse and data lake, to transform your data into features to train models and make predictions.
Write feature pipelines on the platform of your choice or in Hopsworks platform, even running notebooks as orchestrated jobs on airflow.
Create new features either from existing ones or from external data sources. Ingest or leave the data in place storing only metadata.
Retrieve feature data with single millisecond latency from the Hopsworks Feature Store to build model predictions.
The Hopsworks Feature Store is a dual-database platform that includes a low-latency database, for serving the most recent feature data for an entity (e.g. customer), and a scalable database, for storing and accessing large volumes of historical feature values.
The Hopsworks Online Feature Store is based on MySQL Cluster (NDB) that supports low-latency in-memory access to features, five nines availability over different availability zones, and scales 100s of millions of operations/second and up to PBs of data.
The Hopsworks Offline Feature is based on Apache Hive that supports time-travel over historical feature data and enables access to large volumes of feature data in different formats (tfrecord, csv, npy) to train ML models with data scientists’ preferred frameworks (TensorFlow, PyTorch, Scikit-Learn).
Role-based access control within projects allows you to govern which users have “read-only” or “read/write” access to features. In addition, feature engineering jobs run on Hopsworks Feature Store can be assigned either a cluster-wide role or a project-specific role, enabling fine-grained access control and auditing of AWS or Azure services in Hopsworks.
Hopsworks custom metadata support reduces the time spent searching for machine learning artifacts such as feature groups, training data, models, and experiments. Define keywords, and schematized tags to enhance artifact discovery through free-text search on the platform.
Provenance enables the implementation of governance policies, covering challenges such as identifying which features use personally identifiable information, who authorized the use of a given feature in a given model, and which job ingested feature data of interest into the Hopsworks Feature Store.
The Hopsworks Feature Store is a dual-database platform that includes a low-latency database, for serving the most recent feature data for an entity (e.g. customer), and a scalable database, for storing and accessing large volumes of historical feature values.
Any company should be able to easily develop and operate AI even when running at massive scale. We provide an extensive list of integrations to connect the Hopsworks Feature Store to your organization’s data sources and data platforms.
There are virtually no limits on how many features you can store, use, or update. You can start small for free and we will scale with you when you are ready to do it. The Hopsworks Online Feature Store can be scaled to handle millions of concurrent lookups per second.
We are leading the definition of a modern feature store architecture and APIs in the feature store community. We develop solutions for the most challenging feature store use cases together with our customers, making sure you have access to the most advanced technologies available today.
The hopsworks-installer.sh script downloads, configures, and installs Hopsworks. It is typically run interactively, prompting the user about details of what to be installed and where. It can also be run non-interactively (no user prompts) using the ‘-ni’ switch.
wget https://raw.githubusercontent.com/logicalclocks/karamel-chef/2.1/hopsworks-installer.sh
chmod +x hopsworks-installer.sh
./hopsworks-installer.sh
wget https://raw.githubusercontent.com/logicalclocks/karamel-chef/2.1/cloud/hopsworks-cloud-installer.sh
chmod +x hopsworks-cloud-installer.sh
./hopsworks-cloud-installer.sh
Hopsworks Feature Store is available both on AWS and Azure as a managed platform.
You can register for free, without having to enter payment details.
Powered by LogicalClocks © Hopsworks 2021. All rights reserved - Happily using the amazing JAM icons