date:20200304

[incubator-superset] branch master updated: Add option to specify type specific date truncation functions (#9238)

2020-03-04 Thread villebro

This is an automated email from the ASF dual-hosted git repository.

villebro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-superset.git


The following commit(s) were added to refs/heads/master by this push:
 new ef2ebbd  Add option to specify type specific date truncation functions 
(#9238)
ef2ebbd is described below

commit ef2ebbd570524ffede72011803a76eacf1203370
Author: Ville Brofeldt <33317356+ville...@users.noreply.github.com>
AuthorDate: Thu Mar 5 07:25:50 2020 +0200

Add option to specify type specific date truncation functions (#9238)
---
 UPDATING.md |  2 +
 superset/config.py  |  5 +-
 superset/connectors/sqla/models.py  |  4 +-
 superset/db_engine_specs/athena.py  |  2 +-
 superset/db_engine_specs/base.py| 33 -
 superset/db_engine_specs/bigquery.py| 29 
 superset/db_engine_specs/clickhouse.py  |  2 +-
 superset/db_engine_specs/db2.py |  2 +-
 superset/db_engine_specs/dremio.py  |  2 +-
 superset/db_engine_specs/drill.py   |  2 +-
 superset/db_engine_specs/druid.py   |  2 +-
 superset/db_engine_specs/elasticsearch.py   |  2 +-
 superset/db_engine_specs/exasol.py  |  2 +-
 superset/db_engine_specs/hana.py|  2 +-
 superset/db_engine_specs/impala.py  |  2 +-
 superset/db_engine_specs/kylin.py   |  2 +-
 superset/db_engine_specs/mssql.py   |  2 +-
 superset/db_engine_specs/mysql.py   |  2 +-
 superset/db_engine_specs/oracle.py  |  2 +-
 superset/db_engine_specs/pinot.py   | 10 ++--
 superset/db_engine_specs/postgres.py|  2 +-
 superset/db_engine_specs/presto.py  |  2 +-
 superset/db_engine_specs/snowflake.py   |  2 +-
 superset/db_engine_specs/sqlite.py  |  2 +-
 superset/db_engine_specs/teradata.py|  2 +-
 tests/db_engine_specs/base_engine_spec_tests.py |  6 +--
 tests/db_engine_specs/bigquery_tests.py | 61 +
 27 files changed, 109 insertions(+), 79 deletions(-)

diff --git a/UPDATING.md b/UPDATING.md
index f69f237..34537a5 100644
--- a/UPDATING.md
+++ b/UPDATING.md
@@ -23,6 +23,8 @@ assists people when migrating to a new version.
 
 ## Next
 
+* [9238](https://github.com/apache/incubator-superset/pull/9238): the config 
option `TIME_GRAIN_FUNCTIONS` has been renamed to `TIME_GRAIN_EXPRESSIONS` to 
better reflect the content of the dictionary.
+
 * [9218](https://github.com/apache/incubator-superset/pull/9218): SQLite 
connections have been disabled by default
 for analytics databases. You can optionally enable SQLite by setting 
`PREVENT_UNSAFE_DB_CONNECTIONS` to `False`.
 It is not recommended to change this setting, as arbitrary SQLite connections 
can lead to security vulnerabilities.
diff --git a/superset/config.py b/superset/config.py
index 4481a95..443a92d 100644
--- a/superset/config.py
+++ b/superset/config.py
@@ -360,13 +360,14 @@ TIME_GRAIN_BLACKLIST: List[str] = []
 TIME_GRAIN_ADDONS: Dict[str, str] = {}
 
 # Implementation of additional time grains per engine.
+# The column to be truncated is denoted `{col}` in the expression.
 # For example: To implement 2 second time grain on clickhouse engine:
-# TIME_GRAIN_ADDON_FUNCTIONS = {
+# TIME_GRAIN_ADDON_EXPRESSIONS = {
 # 'clickhouse': {
 # 'PT2S': 'toDateTime(intDiv(toUInt32(toDateTime({col})), 2)*2)'
 # }
 # }
-TIME_GRAIN_ADDON_FUNCTIONS: Dict[str, Dict[str, str]] = {}
+TIME_GRAIN_ADDON_EXPRESSIONS: Dict[str, Dict[str, str]] = {}
 
 # ---
 # List of viz_types not allowed in your environment
diff --git a/superset/connectors/sqla/models.py 
b/superset/connectors/sqla/models.py
index 98ba5b2..f249d84 100644
--- a/superset/connectors/sqla/models.py
+++ b/superset/connectors/sqla/models.py
@@ -228,7 +228,9 @@ class TableColumn(Model, BaseColumn):
 col = literal_column(self.expression)
 else:
 col = column(self.column_name)
-time_expr = db.db_engine_spec.get_timestamp_expr(col, pdf, time_grain)
+time_expr = db.db_engine_spec.get_timestamp_expr(
+col, pdf, time_grain, self.type
+)
 return self.table.make_sqla_column_compatible(time_expr, label)
 
 @classmethod
diff --git a/superset/db_engine_specs/athena.py 
b/superset/db_engine_specs/athena.py
index aec6c45..bbdba14 100644
--- a/superset/db_engine_specs/athena.py
+++ b/superset/db_engine_specs/athena.py
@@ -23,7 +23,7 @@ from superset.db_engine_specs.base import BaseEngineSpec
 class AthenaEngineSpec(BaseEngineSpec):
 engine = "awsathena"
 
-_time_grain_functions = {
+_time_grain_expressions = {
 None: "{col}",
 "PT1S": "date_trunc('second', CAST({col} AS TIMESTAMP))",

[incubator-superset] branch master updated: Introducing Inter UI & Fira typefaces (#9207)

2020-03-04 Thread erikrit

This is an automated email from the ASF dual-hosted git repository.

erikrit pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-superset.git


The following commit(s) were added to refs/heads/master by this push:
 new 7d572d9  Introducing Inter UI & Fira typefaces (#9207)
7d572d9 is described below

commit 7d572d9e9621e6c469d6db0afe650e41fcf297b1
Author: Erik Ritter 
AuthorDate: Wed Mar 4 17:32:08 2020 -0800

Introducing Inter UI & Fira typefaces (#9207)

* All the fonts!

* dynamic loading

* folderizing fonts by family

* Shuffling files, getting rid off LESS fanciness

* Fira Mono, consolidating on "specimen" for LESS filenames

* Fira Code

* switch for enabling ligatures

* ligatures off for now

* license

* nixing Fira Mono, using OTF features instead.
---
 LICENSE.txt|   6 +-
 superset-frontend/src/SqlLab/main.less |   1 +
 .../stylesheets/fonts/FiraCode/specimen.less   |  59 ++
 .../fonts/FiraCode/woff/FiraCode-Bold.woff | Bin 0 -> 159636 bytes
 .../fonts/FiraCode/woff/FiraCode-Light.woff| Bin 0 -> 144956 bytes
 .../fonts/FiraCode/woff/FiraCode-Medium.woff   | Bin 0 -> 146332 bytes
 .../fonts/FiraCode/woff/FiraCode-Regular.woff  | Bin 0 -> 146868 bytes
 .../fonts/FiraCode/woff/FiraCode-Retina.woff   | Bin 0 -> 145500 bytes
 .../fonts/FiraCode/woff/FiraCode-VF.woff   | Bin 0 -> 130108 bytes
 .../fonts/FiraCode/woff2/FiraCode-Bold.woff2   | Bin 0 -> 122208 bytes
 .../fonts/FiraCode/woff2/FiraCode-Light.woff2  | Bin 0 -> 113032 bytes
 .../fonts/FiraCode/woff2/FiraCode-Medium.woff2 | Bin 0 -> 113720 bytes
 .../fonts/FiraCode/woff2/FiraCode-Regular.woff2| Bin 0 -> 114456 bytes
 .../fonts/FiraCode/woff2/FiraCode-Retina.woff2 | Bin 0 -> 113088 bytes
 .../fonts/FiraCode/woff2/FiraCode-VF.woff2 | Bin 0 -> 106584 bytes
 .../stylesheets/fonts/InterUI/Inter-Black.woff | Bin 0 -> 140036 bytes
 .../stylesheets/fonts/InterUI/Inter-Black.woff2| Bin 0 -> 104524 bytes
 .../fonts/InterUI/Inter-BlackItalic.woff   | Bin 0 -> 145924 bytes
 .../fonts/InterUI/Inter-BlackItalic.woff2  | Bin 0 -> 109900 bytes
 .../stylesheets/fonts/InterUI/Inter-Bold.woff  | Bin 0 -> 143708 bytes
 .../stylesheets/fonts/InterUI/Inter-Bold.woff2 | Bin 0 -> 107400 bytes
 .../fonts/InterUI/Inter-BoldItalic.woff| Bin 0 -> 149420 bytes
 .../fonts/InterUI/Inter-BoldItalic.woff2   | Bin 0 -> 112580 bytes
 .../stylesheets/fonts/InterUI/Inter-ExtraBold.woff | Bin 0 -> 143552 bytes
 .../fonts/InterUI/Inter-ExtraBold.woff2| Bin 0 -> 107552 bytes
 .../fonts/InterUI/Inter-ExtraBoldItalic.woff   | Bin 0 -> 149196 bytes
 .../fonts/InterUI/Inter-ExtraBoldItalic.woff2  | Bin 0 -> 112876 bytes
 .../fonts/InterUI/Inter-ExtraLight.woff| Bin 0 -> 141864 bytes
 .../fonts/InterUI/Inter-ExtraLight.woff2   | Bin 0 -> 105960 bytes
 .../fonts/InterUI/Inter-ExtraLightItalic.woff  | Bin 0 -> 148688 bytes
 .../fonts/InterUI/Inter-ExtraLightItalic.woff2 | Bin 0 -> 112056 bytes
 .../stylesheets/fonts/InterUI/Inter-Italic.woff| Bin 0 -> 142744 bytes
 .../stylesheets/fonts/InterUI/Inter-Italic.woff2   | Bin 0 -> 107508 bytes
 .../stylesheets/fonts/InterUI/Inter-Light.woff | Bin 0 -> 141528 bytes
 .../stylesheets/fonts/InterUI/Inter-Light.woff2| Bin 0 -> 105640 bytes
 .../fonts/InterUI/Inter-LightItalic.woff   | Bin 0 -> 148436 bytes
 .../fonts/InterUI/Inter-LightItalic.woff2  | Bin 0 -> 111968 bytes
 .../stylesheets/fonts/InterUI/Inter-Medium.woff| Bin 0 -> 142836 bytes
 .../stylesheets/fonts/InterUI/Inter-Medium.woff2   | Bin 0 -> 106720 bytes
 .../fonts/InterUI/Inter-MediumItalic.woff  | Bin 0 -> 149184 bytes
 .../fonts/InterUI/Inter-MediumItalic.woff2 | Bin 0 -> 112504 bytes
 .../stylesheets/fonts/InterUI/Inter-Regular.woff   | Bin 0 -> 134652 bytes
 .../stylesheets/fonts/InterUI/Inter-Regular.woff2  | Bin 0 -> 100124 bytes
 .../stylesheets/fonts/InterUI/Inter-SemiBold.woff  | Bin 0 -> 143248 bytes
 .../stylesheets/fonts/InterUI/Inter-SemiBold.woff2 | Bin 0 -> 107232 bytes
 .../fonts/InterUI/Inter-SemiBoldItalic.woff| Bin 0 -> 149276 bytes
 .../fonts/InterUI/Inter-SemiBoldItalic.woff2   | Bin 0 -> 112552 bytes
 .../stylesheets/fonts/InterUI/Inter-Thin.woff  | Bin 0 -> 136952 bytes
 .../stylesheets/fonts/InterUI/Inter-Thin.woff2 | Bin 0 -> 101408 bytes
 .../fonts/InterUI/Inter-ThinItalic.woff| Bin 0 -> 144412 bytes
 .../fonts/InterUI/Inter-ThinItalic.woff2   | Bin 0 -> 107496 bytes
 .../fonts/InterUI/Inter-italic.var.woff2   | Bin 0 -> 240688 bytes
 .../fonts/InterUI/Inter-roman.var.woff2| Bin 0 -> 226368 bytes
 .../stylesheets/fonts/InterUI/Inter.var.woff2  | Bin 0 -> 316604 bytes

[incubator-superset] branch master updated: fix: choose language link for local dev (#9215)

2020-03-04 Thread erikrit

This is an automated email from the ASF dual-hosted git repository.

erikrit pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-superset.git


The following commit(s) were added to refs/heads/master by this push:
 new b1d83ba  fix: choose language link for local dev (#9215)
b1d83ba is described below

commit b1d83ba8d69d99ca960a865c1e1f6ecf32a4f6cd
Author: Erik Ritter 
AuthorDate: Wed Mar 4 17:08:23 2020 -0800

fix: choose language link for local dev (#9215)

@superset-ui/connection will add protocol and host to an "endpoint" call.

`e.currentTarget.href` may return the full URL instead of the relative
url.
---
 superset-frontend/src/setup/setupApp.ts | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/superset-frontend/src/setup/setupApp.ts 
b/superset-frontend/src/setup/setupApp.ts
index d1c3dd9..ed37e1f 100644
--- a/superset-frontend/src/setup/setupApp.ts
+++ b/superset-frontend/src/setup/setupApp.ts
@@ -67,7 +67,7 @@ export default function setupApp() {
 ) {
   ev.preventDefault();
   SupersetClient.get({
-endpoint: ev.currentTarget.href,
+url: ev.currentTarget.href,
 parseMethod: null,
   }).then(() => {
 location.reload();

[incubator-superset] branch master updated: fix: Oracle fetch_query and datetime conversion (#9240)

2020-03-04 Thread villebro

This is an automated email from the ASF dual-hosted git repository.

villebro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-superset.git


The following commit(s) were added to refs/heads/master by this push:
 new 969bc87  fix: Oracle fetch_query and datetime conversion (#9240)
969bc87 is described below

commit 969bc87431264bb71462ca834b934fd295ed0650
Author: Ville Brofeldt <33317356+ville...@users.noreply.github.com>
AuthorDate: Wed Mar 4 23:43:28 2020 +0200

fix: Oracle fetch_query and datetime conversion (#9240)
---
 superset/db_engine_specs/oracle.py| 15 ---
 tests/db_engine_specs/oracle_tests.py |  5 +
 2 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/superset/db_engine_specs/oracle.py 
b/superset/db_engine_specs/oracle.py
index 2a20d4b..e72eef0 100644
--- a/superset/db_engine_specs/oracle.py
+++ b/superset/db_engine_specs/oracle.py
@@ -17,11 +17,10 @@
 from datetime import datetime
 from typing import Optional
 
-from superset.db_engine_specs.base import LimitMethod
-from superset.db_engine_specs.postgres import PostgresBaseEngineSpec
+from superset.db_engine_specs.base import BaseEngineSpec, LimitMethod
 
 
-class OracleEngineSpec(PostgresBaseEngineSpec):
+class OracleEngineSpec(BaseEngineSpec):
 engine = "oracle"
 limit_method = LimitMethod.WRAP_SQL
 force_column_alias_quotes = True
@@ -44,6 +43,16 @@ class OracleEngineSpec(PostgresBaseEngineSpec):
 tt = target_type.upper()
 if tt == "DATE":
 return f"TO_DATE('{dttm.date().isoformat()}', '-MM-DD')"
+if tt == "DATETIME":
+return f"""TO_DATE('{dttm.isoformat(timespec="seconds")}', 
'-MM-DD"T"HH24:MI:SS')"""  # pylint: disable=line-too-long
 if tt == "TIMESTAMP":
 return 
f"""TO_TIMESTAMP('{dttm.isoformat(timespec="microseconds")}', 
'-MM-DD"T"HH24:MI:SS.ff6')"""  # pylint: disable=line-too-long
 return None
+
+@classmethod
+def epoch_to_dttm(cls) -> str:
+return "TO_DATE('1970-01-01','-MM-DD')+(1/24/60/60)*{col}"
+
+@classmethod
+def epoch_ms_to_dttm(cls) -> str:
+return "TO_DATE('1970-01-01','-MM-DD')+(1/24/60/60/1000)*{col}"
diff --git a/tests/db_engine_specs/oracle_tests.py 
b/tests/db_engine_specs/oracle_tests.py
index 9e39405..09806a0 100644
--- a/tests/db_engine_specs/oracle_tests.py
+++ b/tests/db_engine_specs/oracle_tests.py
@@ -45,6 +45,11 @@ class OracleTestCase(DbEngineSpecTestCase):
 )
 
 self.assertEqual(
+OracleEngineSpec.convert_dttm("DATETIME", dttm),
+"""TO_DATE('2019-01-02T03:04:05', '-MM-DD"T"HH24:MI:SS')""",
+)
+
+self.assertEqual(
 OracleEngineSpec.convert_dttm("TIMESTAMP", dttm),
 """TO_TIMESTAMP('2019-01-02T03:04:05.678900', 
'-MM-DD"T"HH24:MI:SS.ff6')""",
 )

[incubator-superset] branch master updated: fix: share column type matching between model and result set (#9161)

2020-03-04 Thread villebro

This is an automated email from the ASF dual-hosted git repository.

villebro pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-superset.git


The following commit(s) were added to refs/heads/master by this push:
 new 7a91498  fix: share column type matching between model and result set 
(#9161)
7a91498 is described below

commit 7a91498cf1a9e56d4b3d7b3805076137525ea277
Author: Ville Brofeldt <33317356+ville...@users.noreply.github.com>
AuthorDate: Wed Mar 4 16:33:22 2020 +0200

fix: share column type matching between model and result set (#9161)

* Share column type matching between model and result set

* Address comments
---
 superset/common/query_context.py|  2 +-
 superset/connectors/base/models.py  |  6 ++---
 superset/connectors/druid/models.py |  2 +-
 superset/connectors/sqla/models.py  | 37 +++--
 superset/db_engine_specs/base.py| 54 -
 superset/result_set.py  | 11 
 superset/utils/core.py  | 14 +-
 superset/views/core.py  |  1 -
 tests/result_set_tests.py   | 19 ++---
 tests/sqla_models_tests.py  | 48 -
 10 files changed, 151 insertions(+), 43 deletions(-)

diff --git a/superset/common/query_context.py b/superset/common/query_context.py
index edfcfaa..40df7c9 100644
--- a/superset/common/query_context.py
+++ b/superset/common/query_context.py
@@ -123,7 +123,7 @@ class QueryContext:
 
 @staticmethod
 def get_data(  # pylint: disable=invalid-name,no-self-use
-df: pd.DataFrame
+df: pd.DataFrame,
 ) -> List[Dict]:
 return df.to_dict(orient="records")
 
diff --git a/superset/connectors/base/models.py 
b/superset/connectors/base/models.py
index 2f06245..eac6cbb 100644
--- a/superset/connectors/base/models.py
+++ b/superset/connectors/base/models.py
@@ -387,15 +387,15 @@ class BaseColumn(AuditMixinNullable, ImportMixin):
 "DECIMAL",
 "MONEY",
 )
-date_types = ("DATE", "TIME", "DATETIME")
+date_types = ("DATE", "TIME")
 str_types = ("VARCHAR", "STRING", "CHAR")
 
 @property
-def is_num(self) -> bool:
+def is_numeric(self) -> bool:
 return self.type and any(map(lambda t: t in self.type.upper(), 
self.num_types))
 
 @property
-def is_time(self) -> bool:
+def is_temporal(self) -> bool:
 return self.type and any(map(lambda t: t in self.type.upper(), 
self.date_types))
 
 @property
diff --git a/superset/connectors/druid/models.py 
b/superset/connectors/druid/models.py
index ca61a47..517a458 100644
--- a/superset/connectors/druid/models.py
+++ b/superset/connectors/druid/models.py
@@ -527,7 +527,7 @@ class DruidDatasource(Model, BaseDatasource):
 
 @property
 def num_cols(self) -> List[str]:
-return [c.column_name for c in self.columns if c.is_num]
+return [c.column_name for c in self.columns if c.is_numeric]
 
 @property
 def name(self) -> str:  # type: ignore
diff --git a/superset/connectors/sqla/models.py 
b/superset/connectors/sqla/models.py
index 9236bc6..98ba5b2 100644
--- a/superset/connectors/sqla/models.py
+++ b/superset/connectors/sqla/models.py
@@ -145,6 +145,27 @@ class TableColumn(Model, BaseColumn):
 update_from_object_fields = [s for s in export_fields if s not in 
("table_id",)]
 export_parent = "table"
 
+@property
+def is_numeric(self) -> bool:
+db_engine_spec = self.table.database.db_engine_spec
+return db_engine_spec.is_db_column_type_match(
+self.type, utils.DbColumnType.NUMERIC
+)
+
+@property
+def is_string(self) -> bool:
+db_engine_spec = self.table.database.db_engine_spec
+return db_engine_spec.is_db_column_type_match(
+self.type, utils.DbColumnType.STRING
+)
+
+@property
+def is_temporal(self) -> bool:
+db_engine_spec = self.table.database.db_engine_spec
+return db_engine_spec.is_db_column_type_match(
+self.type, utils.DbColumnType.TEMPORAL
+)
+
 def get_sqla_col(self, label: Optional[str] = None) -> Column:
 label = label or self.column_name
 if self.expression:
@@ -489,7 +510,7 @@ class SqlaTable(Model, BaseDatasource):
 
 @property
 def num_cols(self) -> List:
-return [c.column_name for c in self.columns if c.is_num]
+return [c.column_name for c in self.columns if c.is_numeric]
 
 @property
 def any_dttm_col(self) -> Optional[str]:
@@ -809,7 +830,7 @@ class SqlaTable(Model, BaseDatasource):
 is_list_target = op in ("in", "not in")
 eq = self.filter_values_handler(
 flt.get("val"),
-target_column_is_numeric=col_obj.is_num,
+target_column_is_numeric=col_obj.is_numeric,
 is_list_target=is_list_target,

[incubator-superset] branch master updated: Add option to specify type specific date truncation functions (#9238)

[incubator-superset] branch master updated: Introducing Inter UI & Fira typefaces (#9207)

[incubator-superset] branch master updated: fix: choose language link for local dev (#9215)

[incubator-superset] branch master updated: fix: Oracle fetch_query and datetime conversion (#9240)

[incubator-superset] branch master updated: fix: share column type matching between model and result set (#9161)

5 matches

Site Navigation

Mail list logo

Footer information