Hopsworks 0.9.0 Released!

Introducing Hopsworks 0.9.0

Hopsworks 0.9.0 brings the latest features, improvements and bug fixes. It introduces Apache Airflow as-a-service which means users can now create their own workflows from within their familiar environment of a Hopsworks project. This Hopsworks version upgrades Apache Spark to version 2.4.0 which now supports images as datas sources. Additional improvements include a FeatureStore tour to help users get started using their own FeatureStore and an update Jobs service page were jobs can now be fully edited in contrary to them being immutable prior to this version.

Release notes – Hopsworks – Version 0.9.0

Bug

• [HOPSWORKS-546] – Multiple Jupyter notebooks opened for the same user cause PersistenceException
• [HOPSWORKS-727] – Modify influxdb template to account for fqdn
• [HOPSWORKS-846] – Wrong metrics in multiple NN setup
• [HOPSWORKS-865] – Pip search fallback mechanism should catch Exception on the REST request
• [HOPSWORKS-868] – Remove usages of case node[‘platform’] to allow RHEL installation
• [HOPSWORKS-871] – GPUs can get stuck, need nvidia-smi command to restart them
• [HOPSWORKS-876] – Add nvidia/driver_version to metadata.rb
• [HOPSWORKS-888] – featurestore: Infer tf-record-schema does not work with bigint types
• [HOPSWORKS-891] – [Featurestore]: bug, can’t infer join-string in edge-case
• [HOPSWORKS-895] – [Featurestore] bug, get_project_featurestore can return wrong result if project name contains upper-case letters
• [HOPSWORKS-896] – [kagent-chef] Fix broken Anaconda GC command
• [HOPSWORKS-899] – deep learning tour broken after running feature store tour
• [HOPSWORKS-902] – tensorboard.sh script uses hopworks users as group in chmod
• [HOPSWORKS-907] – Crypto material lock gets vanished in CertificateMaterializer
• [HOPSWORKS-910] – Lock down hive-site.xml
• [HOPSWORKS-925] – Hopsworks common dependency on jwt causes persistence unit issues
• [HOPSWORKS-933] – Enable/Disable Airflow microservice for projects

Sub-task

• [HOPSWORKS-752] – Add support for running notebooks (.ipynb) in Job service
• [HOPSWORKS-893] – Remove Zeppelin service icon from Hopsworks

New Feature

• [HOPSWORKS-908] – AirFlow as a Hopsworks microservice

Improvement

• [HOPSWORKS-755] – Fix minor bugs/semantic issues in hops-util-py
• [HOPSWORKS-853] – Feature Store Tour
• [HOPSWORKS-855] – Display readable error in case of naming conflict when sharing datasets
• [HOPSWORKS-870] – Support PIP configuration for internal PyPi servers/proxies
• [HOPSWORKS-872] – Job’s configuration should be editable
• [HOPSWORKS-873] – Change featurestore.create_training_dataset semantics to allow overwrite
• [HOPSWORKS-879] – Add flag to not install epel-repo on Centos/Rhel
• [HOPSWORKS-880] – Make sure umask is 022 before creating envs/pip install
• [HOPSWORKS-887] – Add new variable to enable/disable downloading files
• [HOPSWORKS-894] – InferenceController implementations should use connection pooling
• [HOPSWORKS-906] – Use experiments service in feature store tour
• [HOPSWORKS-921] – Increase timeout for project re-index in epipe

Task

• [HOPSWORKS-838] – Upgrade Spark to 2.4.0
• [HOPSWORKS-866] – Support JobName in addition to JobId as arguments in the Feature Store API
• [HOPSWORKS-867] – Move from Oracle JDK to OpenJDK from the package manager
• [HOPSWORKS-898] – Add rpc size related configs in Livy
• [HOPSWORKS-918] – Bump Hops depdendency in Hopsworks to 2.8.2.7
• [HOPSWORKS-919] – [hopsworks-chef] Create Airflow connection pool in Glassfish on updates
• [HOPSWORKS-922] – (chef) Update kafka attributes