DataSphereStudio - DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling

•    Java

---

DataSphere Studio (DSS for short) is WeDataSphere, a big data platform of WeBank, a self-developed one-stop data application development management portal. Based on Linkis computation middleware, DSS can easily integrate upper-level data application systems, making data application development simple and easy to use.

workflow airflow spark hive hadoop etl kettle hue tableau flink zeppelin griffin azkaban governance davinci visualis supperset linkis scriptis dataworks

Linkis - Linkis helps easily connect to various back-end computation/storage engines(Spark, Python, TiDB

•    Java

---

Linkis helps easily connect to various back-end computation/storage engines

sql spark presto hive storage jdbc rest-api engine impala pyspark udf thrift-server resource-manager jobserver application-manager livy hive-table linkis context-service scriptis

Scriptis - Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent diagnosis

•    Vue

---

Scriptis is for interactive data analysis with script development(SQL, Pyspark, HiveQL), task submission(Spark, Hive), UDF, function, resource management and intelligent diagnosis. Script editor: Support multi-language, auto-completion, syntax highlighting and SQL syntax error-correction.

sql spark hive ide pyspark udf hue zeppelin hql hive-table resouce-management linkis errorcode

Qualitis - Qualitis is a one-stop data quality management platform that supports quality verification, notification, and management for various datasource

•    Java

---

Qualitis is a data quality management platform that supports quality verification, notification, and management for various datasource. It is used to solve various data quality problems caused by data processing. Based on Spring Boot, Qualitis submits quality model task to Linkis platform. It provides functions such as data quality model construction, data quality model execution, data quality verification, reports of data quality generation and so on.

workflow quality compare dss data-quality quality-improvement quality-check linkis datashperestudio data-quality-model

Schedulis - Schedulis is a high performance workflow task scheduling system that supports high availability and multi-tenant financial level features, Linkis computing middleware, and has been integrated into data application development portal DataSphere Studio

•    Java

---

Schedulis is a high performance workflow task scheduling system that supports high availability and multi-tenant financial level features, Linkis computing middleware, and has been integrated into data application development portal DataSphere Studio

workflow scheduler azkaban linkis qualitis datasphere-studio schedulis wedatasphere

Prophecis - Prophecis is a one-stop cloud native machine learning platform.

•    Go

---

Prophecis is a one-stop machine learning platform developed by WeBank. It integrates multiple open-source machine learning frameworks, has the multi tenant management capability of machine learning compute cluster, and provides full stack container deployment and management services for production environment. Prophecis Application Factory：AF is jointly developed by the big data platform team and AI Department of Webank. It is based on QingCloud's open source system KubeSphere custom development, providing CI/CD and DevOps tools, GPU cluster monitoring and warning capabilities.

machine-learning notebook gpu ml multi-tenant-management linkis