Base Command Manager / Bright Cluster Manager Release Notes

Release notes for Bright 8.2

== Edge sites support ==

* Bright Cluster Manager now supports spanning multiple geographic locations
* Support for edge nodes deployment using custom node-installer ISO/USB media
* Edge setup utility to create multiple edge site configurations
* Edge site ISO creation utility

== HPC workload managers ==

* Slurm 18.08.3
* PBS Pro 18.2.2
* PBS Pro Community Edition 18.1.2

== Kubernetes ==

* Support for multiple Kubernetes clusters
* Integration with NVIDIA Docker 2 via NVIDIA Kubernetes plugin for running GPU enabled containers
* Integration with Ceph for persistent storage
** A custom Storage Class makes possible to run pods that consume Ceph volumes
* Improved Kubernetes label management allowing for managing labels also outside of cmdaemon
* New LabelSets for easy assignments of labels to nodes, categories and configuration overlays
* Improved Addons management introducing application groups (KubeAppGroups) in cmdaemon
* Package versions upgrades
** Kubernetes 1.12.3
** Etcd 3.3.10
** CoreDNS 1.2.4
** Heapster 1.5.4
** Helm 2.11.0
** Kubernetes Dashboard 1.10.0
* New Addons
** Calico 3.2.3 (replaces Flannel improving scalability and with Network Policy support)
** NGINX Ingress Controller 0.20.0 (load balancer for external traffic/users)
** NVIDIA device plugin 1.11 (for Kubernetes / NVIDIA Docker 2 integration)
** Metrics Server 0.3.1
* Improved certificate management and HA support

== Container engines ==

* Docker 17.03.2
* NVIDIA Docker 2.0.3
* Singularity 3.0.1

== Container registries ==

* Improved Docker Registry integration based on the GenericRoles
* Harbor 1.6.0
* Docker Compose 1.22.0

== Machine Learning ==

* Spark on Kubernetes
* JupyterHub 0.9.4
* Integrated batch spawners with JupyterHub for UGE, LSF and Slurm
* Added a separate submode in cmsh and BrightView to configure a JupyterHub spawner: batch spawners, systemd, local process of generic spawner

== Ceph ==

* Integration with Ceph Mimic
* FileStore ceph storage backend is now deprecated
* Improved cm-ceph-setup utility
* Ceph builtin dashboard within BrightView
* Ceph builtin Prometheus metrics can be acquired through the Bright View's Monitoring pane or via queries through the Accounting page

== OpenStack ==

* Integration with OpenStack Rocky
* Dropped OpenStack::imageRegistry role (glance-registry service is now deprecated)

== Cloud ==

* Improved cm-cod-aws, cm-cod-azure and cm-cod-openstack
* Improved consolelog output from cloud nodes for AWS, Azure and OpenStack

== cm-scale ==

* New conditional burst triggers: queue length and workload age
* Improved handling of job-exclusive nodes requests
* Support for Slurm logical statements for node features job requests
* Integration with Kubernetes

== Monitoring ==

* Added category / node group / configuration overlay multiplexers
* Added basic REST API with output formats:
** JSON (http://json.org/)
** Universal Binary JSON (http://ubjson.org/)
** CBOR (http://cbor.io/)
** msgpack (https://msgpack.org/)
* Added RPC to remove unused measurables
* Added msgpack, CBOR, UBJSON parsing to monitoring scripts
* PromQL queries printed in table format
* Suppress events from ssh2node and any desired health check, if a node is known to go down

== Linux and Hardware Integration ==

* Support for ARM64 clusters
** Supported distributions: CENTOS7u5, UBUNTU1604, SLES12sp3
** Supported vendors: Huawei, Cavium
* Bare metal installation support for Power8 clusters
* DFARS FIPS support (SLES12 x86_64)
** Allow users to enable FIPS mode from head node installer
** Required components/packages for FIPS installed out of the box
* Added support for CUDA10
* Redfish support
** Power management using Redfish API (Supported vendors: Dell, HPE)
** Metric collection using Redfish API (Supported vendors: Dell, HPE)
** BIOS management using Redfish API (Supported vendors: Dell)
*** Note: This will be available with the first set of updates for Bright 8.2
* Allow for proxy configurations to be done cluster-wide or per node
* Improved GPU overview per node
* Dell BIOS overview and BIOS parameters management improvements

== cmsh ==

* Clone roles between entities (--import option)
* Added --type [,...] selector for all commands in device mode
* Range command for operations on multiple nodes at once
* Added device rename command to rename a list of nodes
* Added command to list routes for one or more nodes
* Added command to list active tcp/udp connections for one or more nodes

== Bright View ==

* Added provisioning node information
* Ability to select disk setup from a template
* Improved wizards

== User Portal ==

* Monitoring for end users (public and private access control)
* Accounting for end users (access control for value of individual)
* User Portal has been ported to Angular framework

== Other ==

* Added by rack / category node selection for boot and provisioning roles
* New DNS role
* New cmsh option to wait until one or more nodes are up
* Improved power on / off / reset timing, scheduling and cancellation
* Added ability to drain provisioning nodes
* Added DNS node-server mapping overview
* Added DHCP node-server mapping overview
* Added provisioning node-server mapping overview
* New all-to-all ping check
* Switched to crypt+sha512 for OpenLDAP password encoding
* Reduced inter-cmdaemon network traffic by factor of 5 by switching to CBOR
* Support for alternative serializations
** JSON (http://json.org/)
** Universal Binary JSON (http://ubjson.org/)
** msgpack (https://msgpack.org/)
* Auditing is enabled by default
* Event logger is enabled by default
* Switch to python script for sampling SMART, with support for raid controllers
* Upgrade hwloc to 1.11.11

== pythoncm ==

* Switch to native pythoncm

== Removed and deprecated features ==

* Integration with Hadoop and associated components has been dropped
* Integration with Mesos and associated components has been dropped
* Integration with Torque work load manager has been dropped
* MIC via MPSS packages installation by Bright has been dropped. UGE RSMAP is still supported by Bright for administrator-installed MPSS MIC installation