Release 3.29.0 (63) - Spark and Python Execution Context (46.47.0 // 1.189.0)

Modified on Sun, 17 Jan 2021 at 01:41 PM

Server Version: 46.47.0 Client Version: 1.189.0

Feature

Enable Usage of Different Spark Execution Context in ONE DATA

Added technical basis for following topics
The spark context can be configured manually in the properties

Python Execution Context

Having scripts as integrated ONE DATA resource, to be able to manage them in ONE DATA
Add functionality to create/read/update scripts
Overview page of existing scripts
Detail page for functions including an editor

Improvements

Loading of Production Lines and Production Line Jobs optimized
Improved performance of statistics calculation
Performance improvements in the user settings
Several smaller UI and visualization improvements

Bugs

Fixed Data Table visualization in Data Table Load Processor for variable usage
Database Connection Load Processor now works with a Hive Connection
"Copy with Header" functionality of Result Table contains line breaks
Add fix for occasionally missing parquet files
Created Model Groups can now be seen by other users
Production Line usage of different versions and variables improved
Fixed the issue of Excessive Spark containers logging

Overall 38 bug fixes.