[jira] [Updated] (DRILL-6430) Drill Should Not Fail If It Sees Deprecated Options Stored In Zookeeper Or Locally
[ https://issues.apache.org/jira/browse/DRILL-6430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pritesh Maker updated DRILL-6430: - Fix Version/s: 1.16.0 > Drill Should Not Fail If It Sees Deprecated Options Stored In Zookeeper Or > Locally > -- > > Key: DRILL-6430 > URL: https://issues.apache.org/jira/browse/DRILL-6430 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Bohdan Kazydub >Priority: Major > Fix For: 1.16.0 > > > This is required for resource management since we will likely remove many > options. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (DRILL-6430) Drill Should Not Fail If It Sees Deprecated Options Stored In Zookeeper Or Locally
[ https://issues.apache.org/jira/browse/DRILL-6430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pritesh Maker reassigned DRILL-6430: Assignee: Bohdan Kazydub (was: Timothy Farkas) > Drill Should Not Fail If It Sees Deprecated Options Stored In Zookeeper Or > Locally > -- > > Key: DRILL-6430 > URL: https://issues.apache.org/jira/browse/DRILL-6430 > Project: Apache Drill > Issue Type: Improvement >Reporter: Timothy Farkas >Assignee: Bohdan Kazydub >Priority: Major > Fix For: 1.16.0 > > > This is required for resource management since we will likely remove many > options. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6866) Upgrade to SqlLine 1.6.0
[
https://issues.apache.org/jira/browse/DRILL-6866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703755#comment-16703755
]
ASF GitHub Bot commented on DRILL-6866:
---
asfgit closed pull request #1556: DRILL-6866: Upgrade to SqlLine 1.6.0
URL: https://github.com/apache/drill/pull/1556
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillSqlLineApplication.java
b/exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillSqlLineApplication.java
index 13d2b3385cd..cc0426b1a65 100644
---
a/exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillSqlLineApplication.java
+++
b/exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillSqlLineApplication.java
@@ -19,10 +19,10 @@
import com.typesafe.config.Config;
import com.typesafe.config.ConfigFactory;
-import jline.console.completer.StringsCompleter;
import org.apache.drill.common.scanner.ClassPathScanner;
import org.apache.drill.common.util.DrillVersionInfo;
import
org.apache.drill.shaded.guava.com.google.common.annotations.VisibleForTesting;
+import org.jline.reader.impl.completer.StringsCompleter;
import sqlline.Application;
import sqlline.CommandHandler;
import sqlline.OutputFormat;
@@ -81,19 +81,24 @@ public Config getConfig() {
}
@Override
- public String getInfoMessage() throws Exception {
+ public String getInfoMessage() {
if (config.hasPath(INFO_MESSAGE_TEMPLATE_CONF)) {
String quote = "";
if (config.hasPath(QUOTES_CONF)) {
List quotes = config.getStringList(QUOTES_CONF);
quote = quotes.get(new Random().nextInt(quotes.size()));
}
- return String.format(config.getString(INFO_MESSAGE_TEMPLATE_CONF),
DrillVersionInfo.getVersion(), quote);
+ return String.format(config.getString(INFO_MESSAGE_TEMPLATE_CONF),
getVersion(), quote);
}
return super.getInfoMessage();
}
+ @Override
+ public String getVersion() {
+return DrillVersionInfo.getVersion();
+ }
+
@Override
public Collection initDrivers() {
if (config.hasPath(DRIVERS_CONF)) {
diff --git a/exec/java-exec/src/main/resources/drill-sqlline.conf
b/exec/java-exec/src/main/resources/drill-sqlline.conf
index e081794a17a..703944d6868 100644
--- a/exec/java-exec/src/main/resources/drill-sqlline.conf
+++ b/exec/java-exec/src/main/resources/drill-sqlline.conf
@@ -83,6 +83,10 @@ drill.sqlline {
opts: {
incremental: false,
isolation: TRANSACTION_NONE,
-maxWidth: 1
+maxWidth: 1,
+maxColumnWidth: 80,
+colorScheme: obsidian,
+# display null values for varchar / char / boolean as null
+nullValue: "null"
}
}
diff --git
a/exec/java-exec/src/test/java/org/apache/drill/exec/client/DrillSqlLineApplicationTest.java
b/exec/java-exec/src/test/java/org/apache/drill/exec/client/DrillSqlLineApplicationTest.java
index 8d3c290ac4e..945df70a8df 100644
---
a/exec/java-exec/src/test/java/org/apache/drill/exec/client/DrillSqlLineApplicationTest.java
+++
b/exec/java-exec/src/test/java/org/apache/drill/exec/client/DrillSqlLineApplicationTest.java
@@ -17,8 +17,10 @@
*/
package org.apache.drill.exec.client;
+import org.apache.drill.common.util.DrillVersionInfo;
import org.junit.BeforeClass;
import org.junit.Test;
+import sqlline.Application;
import sqlline.CommandHandler;
import sqlline.OutputFormat;
import sqlline.SqlLine;
@@ -38,7 +40,7 @@
public class DrillSqlLineApplicationTest {
- private static DrillSqlLineApplication application;
+ private static Application application;
@BeforeClass
public static void init() {
@@ -48,11 +50,16 @@ public static void init() {
}
@Test
- public void testInfoMessage() throws Exception {
+ public void testInfoMessage() {
String infoMessage = application.getInfoMessage();
assertThat(infoMessage, containsString("\"All code is guilty until proven
innocent.\""));
}
+ @Test
+ public void testVersion() {
+assertEquals(DrillVersionInfo.getVersion(), application.getVersion());
+ }
+
@Test
public void testDrivers() {
Collection drivers = application.initDrivers();
@@ -92,8 +99,10 @@ public void testOpts() {
SqlLineOpts opts = application.getOpts(sqlLine);
assertFalse(opts.getIncremental());
assertEquals("TRANSACTION_NONE", opts.getIsolation());
-assertEquals(20_000, opts.getMaxWidth());
+assertEquals(80, opts.getMaxColumnWidth());
assertEquals(200, opts.getTimeout());
+assertEquals("obsidian", opts.getColorScheme());
+assertEquals("null", opts.getNullValue());
}
@Test
diff --git
a/exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillConnect
[jira] [Commented] (DRILL-6792) Find the right probe side fragment to any storage plugin
[
https://issues.apache.org/jira/browse/DRILL-6792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703752#comment-16703752
]
ASF GitHub Bot commented on DRILL-6792:
---
asfgit closed pull request #1504: DRILL-6792: Find the right probe side
fragment wrapper & fix DrillBuf…
URL: https://github.com/apache/drill/pull/1504
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java
b/exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java
index fb465725906..c4d7652f3f0 100644
--- a/exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java
+++ b/exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java
@@ -153,6 +153,10 @@ private ExecConstants() {
public static final IntegerValidator HASHJOIN_BLOOM_FILTER_MAX_SIZE = new
IntegerValidator(HASHJOIN_BLOOM_FILTER_MAX_SIZE_KEY, null);
public static final String HASHJOIN_BLOOM_FILTER_FPP_KEY =
"exec.hashjoin.bloom_filter.fpp";
public static final DoubleValidator HASHJOIN_BLOOM_FILTER_FPP_VALIDATOR =
new RangeDoubleValidator(HASHJOIN_BLOOM_FILTER_FPP_KEY, Double.MIN_VALUE, 1.0,
null);
+ public static final String HASHJOIN_RUNTIME_FILTER_WAITING_ENABLE_KEY =
"exec.hashjoin.runtime_filter.waiting.enable";
+ public static final BooleanValidator HASHJOIN_ENABLE_RUNTIME_FILTER_WAITING
= new BooleanValidator(HASHJOIN_RUNTIME_FILTER_WAITING_ENABLE_KEY, null);
+ public static final String HASHJOIN_RUNTIME_FILTER_MAX_WAITING_TIME_KEY =
"exec.hashjoin.runtime_filter.max.waiting.time";
+ public static final PositiveLongValidator
HASHJOIN_RUNTIME_FILTER_MAX_WAITING_TIME = new
PositiveLongValidator(HASHJOIN_RUNTIME_FILTER_MAX_WAITING_TIME_KEY,
Character.MAX_VALUE, null);
diff --git
a/exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java
b/exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java
index 88c21d9e957..5125f720192 100644
---
a/exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java
+++
b/exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java
@@ -20,8 +20,7 @@
import java.io.IOException;
import java.util.List;
import java.util.concurrent.ExecutorService;
-
-import org.apache.drill.exec.work.filter.RuntimeFilterSink;
+import java.util.concurrent.TimeUnit;
import
org.apache.drill.shaded.guava.com.google.common.annotations.VisibleForTesting;
import org.apache.calcite.schema.SchemaPlus;
import org.apache.drill.common.config.DrillConfig;
@@ -159,18 +158,23 @@ BufferAllocator getNewChildAllocator(final String
operatorName,
@Override
void close();
-
- /**
- * @return
- */
- RuntimeFilterSink getRuntimeFilterSink();
-
/**
* add a RuntimeFilter when the RuntimeFilter receiver belongs to the same
MinorFragment
* @param runtimeFilter
*/
public void addRuntimeFilter(RuntimeFilterWritable runtimeFilter);
+ public RuntimeFilterWritable getRuntimeFilter(long rfIdentifier);
+
+ /**
+ * get the RuntimeFilter with a blocking wait, if the waiting option is
enabled
+ * @param rfIdentifier
+ * @param maxWaitTime
+ * @param timeUnit
+ * @return the RFW or null
+ */
+ public RuntimeFilterWritable getRuntimeFilter(long rfIdentifier, long
maxWaitTime, TimeUnit timeUnit);
+
interface ExecutorState {
/**
* Tells individual operations whether they should continue. In some
cases, an external event (typically cancellation)
diff --git
a/exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContextImpl.java
b/exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContextImpl.java
index 6e40466e8e5..b740c927daf 100644
---
a/exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContextImpl.java
+++
b/exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContextImpl.java
@@ -21,7 +21,12 @@
import java.util.List;
import java.util.Map;
import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutorService;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.locks.Condition;
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReentrantLock;
import org.apache.calcite.schema.SchemaPlus;
import org.apache.drill.common.config.DrillConfig;
@@ -60,8 +65,6 @@
import org.apache.drill.exec.testing.ExecutionControls;
import org.apache.drill.exec.util.ImpersonationUtil;
import org.apache.drill.exec.work.batch.IncomingBuffers;
-
-import org.apache.drill.exec.work.filter.RuntimeFilterSink;
import org.apache.drill.shaded.guava.com.google.common.base.Function;
import org.apache.dril
[jira] [Commented] (DRILL-6039) drillbit.sh graceful_stop does not wait for fragments to complete before stopping the drillbit
[
https://issues.apache.org/jira/browse/DRILL-6039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703751#comment-16703751
]
ASF GitHub Bot commented on DRILL-6039:
---
asfgit closed pull request #1536: DRILL-6039: Fixed drillbit.sh script to do
graceful shutdown
URL: https://github.com/apache/drill/pull/1536
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/distribution/src/resources/drill-config.sh
b/distribution/src/resources/drill-config.sh
index d23788b006a..a4686c50354 100644
--- a/distribution/src/resources/drill-config.sh
+++ b/distribution/src/resources/drill-config.sh
@@ -334,6 +334,7 @@ fi
# provided in drill-env.sh.
export DRILL_PID_DIR=${DRILL_PID_DIR:-$DRILL_HOME}
+export GRACEFUL_SIGFILE=${GRACEFUL_SIGFILE:-"graceful"}
# Prepare log file prefix and the main Drillbit log file.
diff --git a/distribution/src/resources/drillbit.sh
b/distribution/src/resources/drillbit.sh
index 88d56c8a14f..5ad87b15cdb 100755
--- a/distribution/src/resources/drillbit.sh
+++ b/distribution/src/resources/drillbit.sh
@@ -87,6 +87,7 @@ export args
# Set default scheduling priority
DRILL_NICENESS=${DRILL_NICENESS:-0}
+GRACEFUL_FILE=$DRILL_PID_DIR/$GRACEFUL_SIGFILE
waitForProcessEnd()
{
@@ -94,11 +95,19 @@ waitForProcessEnd()
commandName=$2
kill_drillbit=$3
processedAt=`date +%s`
+ triggered_shutdown=false
origcnt=${DRILL_STOP_TIMEOUT:-120}
while kill -0 $pidKilled > /dev/null 2>&1;
do
echo -n "."
sleep 1;
+ #Incase of graceful shutdown, create graceful file and wait till the
process ends.
+ if [ "$kill_drillbit" = false ]; then
+ if [ "$triggered_shutdown" = false ]; then
+ touch $GRACEFUL_FILE
+ triggered_shutdown=true
+ fi
+ fi
if [ "$kill_drillbit" = true ] ; then
# if process persists more than $DRILL_STOP_TIMEOUT (default 120 sec)
no mercy
if [ $(( `date +%s` - $processedAt )) -gt $origcnt ]; then
@@ -125,6 +134,15 @@ check_before_start()
exit 1
fi
fi
+ #remove any previous uncleaned graceful file
+ if [ -f "$GRACEFUL_FILE" ]; then
+rm $GRACEFUL_FILE
+rm_status=$?
+if [ $rm_status -ne 0 ];then
+echo "Error: Failed to remove $GRACEFUL_FILE!"
+exit $rm_status
+fi
+ fi
}
check_after_start(){
@@ -204,7 +222,9 @@ stop_bit ( )
if kill -0 $pidToKill > /dev/null 2>&1; then
echo "Stopping $command"
echo "`date` Terminating $command pid $pidToKill" >> "$DRILLBIT_LOG_PATH"
- kill $pidToKill > /dev/null 2>&1
+ if [ $kill_drillbit = true ]; then
+kill $pidToKill > /dev/null 2>&1
+ fi
waitForProcessEnd $pidToKill $command $kill_drillbit
retval=0
else
diff --git
a/exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java
b/exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java
index dd1c5f19faf..a0c63ab6dcc 100644
--- a/exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java
+++ b/exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java
@@ -17,6 +17,13 @@
*/
package org.apache.drill.exec.server;
+import java.io.IOException;
+import java.nio.file.FileSystems;
+import java.nio.file.Path;
+import java.nio.file.StandardWatchEventKinds;
+import java.nio.file.WatchEvent;
+import java.nio.file.WatchKey;
+import java.nio.file.WatchService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
@@ -90,6 +97,8 @@
private DrillbitStateManager stateManager;
private boolean quiescentMode;
private boolean forcefulShutdown = false;
+ GracefulShutdownThread gracefulShutdownThread;
+ private boolean interruptPollShutdown = true;
public void setQuiescentMode(boolean quiescentMode) {
this.quiescentMode = quiescentMode;
@@ -212,6 +221,8 @@ public void run() throws Exception {
drillbitContext.startRM();
Runtime.getRuntime().addShutdownHook(new ShutdownThread(this, new
StackTrace()));
+gracefulShutdownThread = new GracefulShutdownThread(this, new
StackTrace());
+gracefulShutdownThread.start();
logger.info("Startup completed ({} ms).",
w.elapsed(TimeUnit.MILLISECONDS));
}
@@ -291,6 +302,11 @@ public synchronized void close() {
logger.info("Shutdown completed ({} ms).",
w.elapsed(TimeUnit.MILLISECONDS) );
stateManager.setState(DrillbitState.SHUTDOWN);
+// Interrupt GracefulShutdownThread since Drillbit close is not called
from it.
+if (interruptPollShutdown) {
+ gracefulShutdownThread.interrupt();
+}
+
}
private void javaPropertiesToSystemOptions() {
@@ -335,6 +351,55 @@ private void javaPropertiesToSyste
[jira] [Commented] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703753#comment-16703753
]
ASF GitHub Bot commented on DRILL-6863:
---
asfgit closed pull request #1557: DRILL-6863: Drop table is not working if path
within workspace starts…
URL: https://github.com/apache/drill/pull/1557
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/SqlConverter.java
b/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/SqlConverter.java
index 51aad406c48..154bf8cbb41 100644
---
a/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/SqlConverter.java
+++
b/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/SqlConverter.java
@@ -82,6 +82,7 @@
import org.apache.drill.exec.planner.physical.DrillDistributionTraitDef;
import org.apache.drill.exec.planner.physical.PlannerSettings;
import org.apache.drill.exec.rpc.user.UserSession;
+import org.apache.drill.exec.store.dfs.FileSelection;
import static org.apache.calcite.util.Static.RESOURCE;
import org.apache.drill.shaded.guava.com.google.common.base.Joiner;
@@ -624,7 +625,8 @@ void disallowTemporaryTables() {
private List getTemporaryNames(List names) {
if (mightBeTemporaryTable(names, session.getDefaultSchemaPath(),
drillConfig)) {
-String temporaryTableName =
session.resolveTemporaryTableName(names.get(names.size() - 1));
+String tableName =
FileSelection.removeLeadingSlash(names.get(names.size() - 1));
+String temporaryTableName =
session.resolveTemporaryTableName(tableName);
if (temporaryTableName != null) {
List temporaryNames = new
ArrayList<>(SchemaUtilites.getSchemaPathAsList(temporarySchema));
temporaryNames.add(temporaryTableName);
diff --git
a/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/CreateTableHandler.java
b/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/CreateTableHandler.java
index 928a849884a..82a111a1d97 100644
---
a/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/CreateTableHandler.java
+++
b/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/CreateTableHandler.java
@@ -55,6 +55,7 @@
import org.apache.drill.exec.planner.sql.SchemaUtilites;
import org.apache.drill.exec.planner.sql.parser.SqlCreateTable;
import org.apache.drill.exec.store.AbstractSchema;
+import org.apache.drill.exec.store.dfs.FileSelection;
import org.apache.drill.exec.util.Pointer;
import org.apache.drill.exec.work.foreman.ForemanSetupException;
import org.apache.drill.exec.work.foreman.SqlUnsupportedException;
@@ -72,7 +73,7 @@ public CreateTableHandler(SqlHandlerConfig config,
Pointer textPlan) {
@Override
public PhysicalPlan getPlan(SqlNode sqlNode) throws ValidationException,
RelConversionException, IOException, ForemanSetupException {
final SqlCreateTable sqlCreateTable = unwrap(sqlNode,
SqlCreateTable.class);
-final String originalTableName = sqlCreateTable.getName();
+final String originalTableName =
FileSelection.removeLeadingSlash(sqlCreateTable.getName());
final ConvertedRelNode convertedRelNode =
validateAndConvert(sqlCreateTable.getQuery());
final RelDataType validatedRowType =
convertedRelNode.getValidatedRowType();
diff --git
a/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DropTableHandler.java
b/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DropTableHandler.java
index c17ac20840a..4f1a7596e3b 100644
---
a/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DropTableHandler.java
+++
b/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DropTableHandler.java
@@ -34,6 +34,7 @@
import org.apache.drill.exec.planner.sql.parser.SqlDropTable;
import org.apache.drill.exec.rpc.user.UserSession;
import org.apache.drill.exec.store.AbstractSchema;
+import org.apache.drill.exec.store.dfs.FileSelection;
// SqlHandler for dropping a table.
public class DropTableHandler extends DefaultSqlHandler {
@@ -56,7 +57,7 @@ public DropTableHandler(SqlHandlerConfig config) {
@Override
public PhysicalPlan getPlan(SqlNode sqlNode) throws ValidationException,
RelConversionException, IOException {
SqlDropTable dropTableNode = ((SqlDropTable) sqlNode);
-String originalTableName = dropTableNode.getName();
+String originalTableName =
FileSelection.removeLeadingSlash(dropTableNode.getName());
SchemaPlus defaultSchema = config.getConverter().getDefaultSchema();
List tableSchema = dropTableNode.getSchema
[jira] [Commented] (DRILL-6867) WebUI Query editor cursor position
[
https://issues.apache.org/jira/browse/DRILL-6867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703754#comment-16703754
]
ASF GitHub Bot commented on DRILL-6867:
---
asfgit closed pull request #1551: DRILL-6867: WebUI Query editor cursor position
URL: https://github.com/apache/drill/pull/1551
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/exec/java-exec/src/main/resources/rest/query/query.ftl
b/exec/java-exec/src/main/resources/rest/query/query.ftl
index 00fdd9e6fe4..fd3e8bd6dd3 100644
--- a/exec/java-exec/src/main/resources/rest/query/query.ftl
+++ b/exec/java-exec/src/main/resources/rest/query/query.ftl
@@ -103,7 +103,7 @@
editor.$blockScrolling = "Infinity";
//CSS Formatting
document.getElementById('query-editor-format').style.fontSize='13px';
-document.getElementById('query-editor-format').style.fontFamily='courier';
+
document.getElementById('query-editor-format').style.fontFamily='courier,monospace';
document.getElementById('query-editor-format').style.lineHeight='1.5';
document.getElementById('query-editor-format').style.width='98%';
document.getElementById('query-editor-format').style.margin='auto';
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> WebUI Query editor cursor position
> --
>
> Key: DRILL-6867
> URL: https://issues.apache.org/jira/browse/DRILL-6867
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0
>Reporter: Oleg Zinoviev
>Assignee: Oleg Zinoviev
>Priority: Minor
> Labels: ready-to-commit
> Fix For: 1.15.0
>
> Attachments: query.png
>
>
> In Firefox (at least on Ununtu 18.10), the position of the cursor does not
> match the text when editing the SQL-query
> !query.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6492) Make storage plugins names case insensitive
[ https://issues.apache.org/jira/browse/DRILL-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703643#comment-16703643 ] Bridget Bevens commented on DRILL-6492: --- Hi [~arina], I've updated this page with the case sensitivity changes: https://drill.apache.org/docs/lexical-structure/ Please have a look and let me know if I missed anything. Thanks, Bridget > Make storage plugins names case insensitive > --- > > Key: DRILL-6492 > URL: https://issues.apache.org/jira/browse/DRILL-6492 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.13.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva >Priority: Major > Labels: doc-impacting, ready-to-commit > Fix For: 1.15.0 > > > Storage plugin names to be case insensitive (DFS vs dfs, INFORMATION_SCHEMA > vs information_schema). > Workspace (schemas) names to be case insensitive (ROOT vs root, TMP vs tmp). > Even if user has two directories /TMP and /tmp, he can create two workspaces > but not both with tmp name. For example, tmp vs tmp_u. > Table names case sensitivity are treated per plugin. For example, system > plugins (information_schema, sys) table names (views, tables) should be case > insensitive. Actually, currently for sys plugin table names are case > insensitive, information_schema table names are case sensitive. That needs to > be synchronized. For file system plugins table names must be case sensitive, > since under table name we imply directory / file name and their case > sensitivity depends on file system. > *Documentation* > https://drill.apache.org/docs/lexical-structure/ should be updated with > relevant information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6179) Added pcapng-format support
[ https://issues.apache.org/jira/browse/DRILL-6179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bridget Bevens updated DRILL-6179: -- Labels: doc-complete ready-to-commit (was: doc-impacting ready-to-commit) > Added pcapng-format support > --- > > Key: DRILL-6179 > URL: https://issues.apache.org/jira/browse/DRILL-6179 > Project: Apache Drill > Issue Type: New Feature >Affects Versions: 1.13.0 >Reporter: Vlad >Assignee: Vlad >Priority: Major > Labels: doc-complete, ready-to-commit > Fix For: 1.15.0 > > > The _PCAP Next Generation Dump File Format_ (or pcapng for short) [1] is an > attempt to overcome the limitations of the currently widely used (but > limited) libpcap format. > At a first level, it is desirable to query and filter by source and > destination IP and port, and src/dest mac addreses or by protocol. Beyond > that, however, it would be very useful to be able to group packets by TCP > session and eventually to look at packet contents. > Initial work is available at > https://github.com/mapr-demos/drill/tree/pcapng_dev > [1] https://pcapng.github.io/pcapng/ > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6179) Added pcapng-format support
[ https://issues.apache.org/jira/browse/DRILL-6179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703659#comment-16703659 ] Bridget Bevens commented on DRILL-6179: --- Added the pcapng format to the list of supported formats on this page: https://drill.apache.org/docs/plugin-configuration-basics/ Setting doc label to doc-complete. Please let me know if this requires additional documentation. Thanks, Bridget > Added pcapng-format support > --- > > Key: DRILL-6179 > URL: https://issues.apache.org/jira/browse/DRILL-6179 > Project: Apache Drill > Issue Type: New Feature >Affects Versions: 1.13.0 >Reporter: Vlad >Assignee: Vlad >Priority: Major > Labels: doc-impacting, ready-to-commit > Fix For: 1.15.0 > > > The _PCAP Next Generation Dump File Format_ (or pcapng for short) [1] is an > attempt to overcome the limitations of the currently widely used (but > limited) libpcap format. > At a first level, it is desirable to query and filter by source and > destination IP and port, and src/dest mac addreses or by protocol. Beyond > that, however, it would be very useful to be able to group packets by TCP > session and eventually to look at packet contents. > Initial work is available at > https://github.com/mapr-demos/drill/tree/pcapng_dev > [1] https://pcapng.github.io/pcapng/ > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6492) Make storage plugins names case insensitive
[ https://issues.apache.org/jira/browse/DRILL-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bridget Bevens updated DRILL-6492: -- Labels: doc-complete ready-to-commit (was: doc-impacting ready-to-commit) > Make storage plugins names case insensitive > --- > > Key: DRILL-6492 > URL: https://issues.apache.org/jira/browse/DRILL-6492 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.13.0 >Reporter: Arina Ielchiieva >Assignee: Arina Ielchiieva >Priority: Major > Labels: doc-complete, ready-to-commit > Fix For: 1.15.0 > > > Storage plugin names to be case insensitive (DFS vs dfs, INFORMATION_SCHEMA > vs information_schema). > Workspace (schemas) names to be case insensitive (ROOT vs root, TMP vs tmp). > Even if user has two directories /TMP and /tmp, he can create two workspaces > but not both with tmp name. For example, tmp vs tmp_u. > Table names case sensitivity are treated per plugin. For example, system > plugins (information_schema, sys) table names (views, tables) should be case > insensitive. Actually, currently for sys plugin table names are case > insensitive, information_schema table names are case sensitive. That needs to > be synchronized. For file system plugins table names must be case sensitive, > since under table name we imply directory / file name and their case > sensitivity depends on file system. > *Documentation* > https://drill.apache.org/docs/lexical-structure/ should be updated with > relevant information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-540) Allow querying hive views in Drill
[
https://issues.apache.org/jira/browse/DRILL-540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Guzenko updated DRILL-540:
---
Description:
Currently hive views cannot be queried from drill.
This Jira aims to add support for Hive views in Drill.
*Implementation details:*
# Drill persists it's views metadata in file with suffix .view.drill using
json format. For example:
{noformat}
{
"name" : "view_from_calcite_1_4",
"sql" : "SELECT * FROM `cp`.`store.json`WHERE `store_id` = 0",
"fields" : [ {
"name" : "*",
"type" : "ANY",
"isNullable" : true
} ],
"workspaceSchemaPath" : [ "dfs", "tmp" ]
}
{noformat}
Later Drill parses the metadata and uses it to treat view names in SQL as a
subquery.
2. In Apache Hive metadata about views is stored in similar way to
tables. Below is example from metastore.TBLS :
{noformat}
TBL_ID |CREATE_TIME |DB_ID |LAST_ACCESS_TIME |OWNER |RETENTION |SD_ID |TBL_NAME
|TBL_TYPE |VIEW_EXPANDED_TEXT |
---||--|-|--|--|--|--|--|---|
2 |1542111078 |1 |0|mapr |0 |2 |cview
|VIRTUAL_VIEW |SELECT COUNT(*) FROM `default`.`customers` |
{noformat}
3. So in Hive metastore views are considered as tables of special type.
And main benefit is that we also have expanded SQL definition of views (just
like in view.drill files). Also reading of the metadata is already implemented
in Drill with help of thrift Metastore API.
4. To enable querying of Hive views we'll reuse existing code for Drill
views as much as possible. First in *_HiveSchemaFactory.getDrillTable_* for
_*HiveReadEntry*_ we'll convert the metadata to instance of _*View*_ (_which is
actually model for data persisted in .view.drill files_) and then based on this
instance return new _*DrillViewTable*_. Using this approach drill will handle
hive views the same way as if it was initially defined in Drill and persisted
in .view.drill file.
5. For conversion of Hive types: from _*FieldSchema*_ to _*RelDataType*_
we'll reuse existing code from _*DrillHiveTable*_, so the conversion
functionality will be extracted and used for both (table and view) fields type
conversions.
*Security implications*
Consider simple example case where we have users,
{code:java}
user0 user1 user2
\ /
group12
{code}
and sample db where object names contains user or group who should access them
{code:java}
db_all
tbl_user0
vw_user0
tbl_group12
vw_group12
{code}
There are two Hive authorization modes supported by Drill - SQL Standart and
Strorage Based authorization. For SQL Standart authorization permissions were
granted using SQL:
{code:java}
SET ROLE admin;
GRANT SELECT ON db_all.tbl_user0 TO USER user0;
GRANT SELECT ON db_all.vw_user0 TO USER user0;
CREATE ROLE group12;
GRANT ROLE group12 TO USER user1;
GRANT ROLE group12 TO USER user2;
GRANT SELECT ON db_all.tbl_group12 TO ROLE group12;
GRANT SELECT ON db_all.vw_group12 TO ROLE group12;
{code}
And for Storage based authorization permissions were granted using commands:
{code:java}
hadoop fs -chown user0:user0 /user/hive/warehouse/db_all.db/tbl_user0
hadoop fs -chmod 700 /user/hive/warehouse/db_all.db/tbl_user0
hadoop fs -chmod 750 /user/hive/warehouse/db_all.db/tbl_group12
hadoop fs -chown user1:group12 /user/hive/warehouse/db_all.db/tbl_group12{code}
Then the following table shows us results of queries for both authorization
models.
*SQL
Standart Storage Based Authorization*
||SQL||user0||user1||user2|| ||user0||user1||user2||
|SHOW TABLES IN hive.db_all;| all| all| all| |Accessibe tables + all
views|Accessibe tables + all views|Accessibe tables + all views|
|SELECT * FROM hive.db_all.tbl_user0;| (/)| (x)| (x)| | (/)|
(x)| (x)|
|SELECT * FROM hive.db_all.vw_user0;| (/)| (x)| (x)| | (/)|
(x)| (x)|
|SELECT * FROM hive.db_all.tbl_group12;| (x)| (/)| (/)| | (x)|
(/)| (/)|
|SELECT * FROM hive.db_all.vw_group12;| (x)| (/)| (/)| | (x)|
(/)| (/)|
|SELECT * FROM INFORMATION_SCHEMA.`TABLES`
WHERE TABLE_SCHEMA = 'hive.db_all';| all| all| all| |Accessibe tables +
all views|Accessibe tables + all views|Accessibe tables + all views|
|DESCRIBE hive.db_all.tbl_user0;| (/)| (x)| (x)| | (/)|
(x)| (x)|
|DESCRIBE hive.db_all.vw_user0;| (/)| (x) | (x)| | (/)|
(/)| (/)|
|DESCRIBE hive.db_all.tbl_group12;| (x)| (/)| (/)| | (x)|
(/) | (/)|
|DESCRIBE hive.db_all.vw_group12;| (x)| (/)| (/)| | (/)|
(/)| (/)|
(!) *Warning:* Because views in Hiv
[jira] [Updated] (DRILL-540) Allow querying hive views in Drill
[
https://issues.apache.org/jira/browse/DRILL-540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Guzenko updated DRILL-540:
---
Description:
Currently hive views cannot be queried from drill.
This Jira aims to add support for Hive views in Drill.
*Implementation details:*
# Drill persists it's views metadata in file with suffix .view.drill using
json format. For example:
{noformat}
{
"name" : "view_from_calcite_1_4",
"sql" : "SELECT * FROM `cp`.`store.json`WHERE `store_id` = 0",
"fields" : [ {
"name" : "*",
"type" : "ANY",
"isNullable" : true
} ],
"workspaceSchemaPath" : [ "dfs", "tmp" ]
}
{noformat}
Later Drill parses the metadata and uses it to treat view names in SQL as a
subquery.
2. In Apache Hive metadata about views is stored in similar way to
tables. Below is example from metastore.TBLS :
{noformat}
TBL_ID |CREATE_TIME |DB_ID |LAST_ACCESS_TIME |OWNER |RETENTION |SD_ID |TBL_NAME
|TBL_TYPE |VIEW_EXPANDED_TEXT |
---||--|-|--|--|--|--|--|---|
2 |1542111078 |1 |0|mapr |0 |2 |cview
|VIRTUAL_VIEW |SELECT COUNT(*) FROM `default`.`customers` |
{noformat}
3. So in Hive metastore views are considered as tables of special type.
And main benefit is that we also have expanded SQL definition of views (just
like in view.drill files). Also reading of the metadata is already implemented
in Drill with help of thrift Metastore API.
4. To enable querying of Hive views we'll reuse existing code for Drill
views as much as possible. First in *_HiveSchemaFactory.getDrillTable_* for
_*HiveReadEntry*_ we'll convert the metadata to instance of _*View*_ (_which is
actually model for data persisted in .view.drill files_) and then based on this
instance return new _*DrillViewTable*_. Using this approach drill will handle
hive views the same way as if it was initially defined in Drill and persisted
in .view.drill file.
5. For conversion of Hive types: from _*FieldSchema*_ to _*RelDataType*_
we'll reuse existing code from _*DrillHiveTable*_, so the conversion
functionality will be extracted and used for both (table and view) fields type
conversions.
*Security implications*
Consider simple example case where we have users,
{code:java}
user0 user1 user2
\ /
group12
{code}
and sample db where object names contains user or group who should access them
{code:java}
db_all
tbl_user0
vw_user0
tbl_group12
vw_group12
{code}
There are two Hive authorization modes supported by Drill - SQL Standart and
Strorage Based authorization.
For SQL Standart authorization permissions were granted using SQL:
{code:java}
SET ROLE admin;
GRANT SELECT ON db_all.tbl_user0 TO USER user0;
GRANT SELECT ON db_all.vw_user0 TO USER user0;
CREATE ROLE group12;
GRANT ROLE group12 TO USER user1;
GRANT ROLE group12 TO USER user2;
GRANT SELECT ON db_all.tbl_group12 TO ROLE group12;
GRANT SELECT ON db_all.vw_group12 TO ROLE group12;
{code}
And for Storage based authorization permissions were granted using commands:
{code:java}
hadoop fs -chown user0:user0 /user/hive/warehouse/db_all.db/tbl_user0
hadoop fs -chmod 700 /user/hive/warehouse/db_all.db/tbl_user0
hadoop fs -chmod 750 /user/hive/warehouse/db_all.db/tbl_group12
hadoop fs -chown user1:group12 /user/hive/warehouse/db_all.db/tbl_group12{code}
Then the following table shows us results of queries for both authorization
models.
*SQL
Standart Storage Based Authorization*
||SQL||user0||user1||user2|| ||user0||user1||user2||
|SHOW TABLES IN hive.db_all;| all| all| all| |Accessibe tables + all
views|Accessibe tables + all views|Accessibe tables + all views|
|SELECT * FROM hive.db_all.tbl_user0;| (/)| (x)| (x)| | (/)|
(x)| (x)|
|SELECT * FROM hive.db_all.vw_user0;| (/)| (x)| (x)| | (/)|
(x)| (x)|
|SELECT * FROM hive.db_all.tbl_group12;| (x)| (/)| (/)| | (x)|
(/)| (/)|
|SELECT * FROM hive.db_all.vw_group12;| (x)| (/)| (/)| | (x)|
(/)| (/)|
|SELECT * FROM INFORMATION_SCHEMA.`TABLES`
WHERE TABLE_SCHEMA = 'hive.db_all';| all| all| all| |Accessibe tables +
all views|Accessibe tables + all views|Accessibe tables + all views|
|DESCRIBE hive.db_all.tbl_user0;| (/)| (x)| (x)| | (/)|
(x)| (x)|
|DESCRIBE hive.db_all.vw_user0;| (/)| (x) | (x)| | (/)|
(/)| (/)|
|DESCRIBE hive.db_all.tbl_group12;| (x)| (/)| (/)| | (x)|
(/) | (/)|
|DESCRIBE hive.db_all.vw_group12;| (x)| (/)| (/)| | (/)|
(/)| (/)|
(!) *Warnin
[jira] [Updated] (DRILL-6790) Updating SET and ALTER SYSTEM doc pages
[
https://issues.apache.org/jira/browse/DRILL-6790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6790:
Fix Version/s: (was: 1.15.0)
1.16.0
> Updating SET and ALTER SYSTEM doc pages
> ---
>
> Key: DRILL-6790
> URL: https://issues.apache.org/jira/browse/DRILL-6790
> Project: Apache Drill
> Issue Type: Task
> Components: Documentation
>Affects Versions: 1.14.0
>Reporter: Vitalii Diravka
>Assignee: Bridget Bevens
>Priority: Major
> Labels: doc-impacting
> Fix For: 1.16.0
>
>
> It is necessary to update old queries related to "sys.options" table in SET
> [https://drill.apache.org/docs/set/] and ALTER SYSTEM
> [https://drill.apache.org/docs/alter-system/] drill doc pages:
> {code}
> 0: jdbc:drill:zk=local> SELECT name, type, bool_val FROM sys.options WHERE
> type = 'SESSION' order by name;
> 18:04:32.669 [2440997e-a5e7-4458-b7f9-661df830db5f:foreman] ERROR
> o.a.calcite.runtime.CalciteException -
> org.apache.calcite.sql.validate.SqlValidatorException: Column 'type' not
> found in any table
> 18:04:32.670 [2440997e-a5e7-4458-b7f9-661df830db5f:foreman] ERROR
> o.a.calcite.runtime.CalciteException -
> org.apache.calcite.runtime.CalciteContextException: From line 1, column 52 to
> line 1, column 55: Column 'type' not found in any table
> Error: VALIDATION ERROR: From line 1, column 52 to line 1, column 55: Column
> 'type' not found in any table
> [Error Id: ffb96168-50e6-4d38-baa1-09429f3f191b on vitalii-pc:31010]
> (state=,code=0)
> {code}
> DRILL-5723 introduces new fields for "sys.options" system table.
> The "type" field could be replaced with "optionScope":
> {code}
> 0: jdbc:drill:zk=local> SELECT * FROM sys.options WHERE optionScope in
> ('SYSTEM','SESSION') order by name;
> +-+---+---++--+--+-+
> | name | kind | accessibleScopes
> | val | status | optionScope |
> description |
> +-+---+---++--+--+-+
> | drill.exec.functions.cast_empty_string_to_null | BIT | ALL
> | true | CHANGED | SESSION | In a text file, treat empty fields as
> NULL values instead of empty string. |
> | drill.exec.hashagg.fallback.enabled | BIT | ALL
> | false | DEFAULT | SYSTEM |
>|
> +-+---+---++--+--+-+
> 2 rows selected (0.113 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-4587) Document Drillbit launch options
[ https://issues.apache.org/jira/browse/DRILL-4587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-4587: Fix Version/s: (was: 1.15.0) 1.16.0 > Document Drillbit launch options > > > Key: DRILL-4587 > URL: https://issues.apache.org/jira/browse/DRILL-4587 > Project: Apache Drill > Issue Type: Improvement > Components: Documentation >Reporter: Paul Rogers >Assignee: Bridget Bevens >Priority: Major > Fix For: 1.16.0 > > > Drill provides the drillbit.sh script to launch Drill. When Drill is run in > production environments, or when managed by a tool such as Mesos or YARN, > customers have many options to customize the launch options. We should > document this information as below. > The user can configure Drill launch in one of four ways, depending on their > needs. > 1. Using the properties in drill-override.conf. Sets only startup and runtime > properties. All drillbits should use a copy of the file so that properties > set here apply to all drill bits and to client applications. > 2. By setting environment variables prior to launching Drill. See the list > below. Use this to customize properties per drill-bit, such as for setting > port numbers. This option is useful when launching Drill from a tool such as > Mesos or YARN. > 3. By setting environment variables in $DRILL_HOME/conf/drill-env.sh. See the > list below. This script is intended to be unique to each node and is another > way to customize properties for this one node. > 4. In Drill 1.7 and later, the administrator can set Drill configuration > options directly on the launch command as shown below. This option is also > useful when launching Drill from a tool such as YARN or Mesos. Options are of > the form: > $ drillbit.sh start -Dvariable=value > For example, to control the HTTP port: > $ drillbit.sh start -Ddrill.exec.http.port=8099 > Properties are of three types. > 1. Launch-only properties: those that can be set only through environment > variables (such as JAVA_HOME.) > 2. Drill startup properties which can be set in the locations detailed below. > 3. Drill runtime properties which are set in drill-override.conf also via SQL. > Drill startup propeties can be set in a number of locations. Those listed > later take precedence over those listed earlier. > 1. Drill-override.conf as identified by DRILL_CONF_DIR or its default. > 2. Set in the environment using DRILL_JAVA_OPTS or DRILL_DRILLBIT_JAVA_OPTS. > 3. Set in drill-env.sh using the above two variables. > 4. Set on the drill.bit command line as explained above. (Drill 1.7 and > later.) > You can see the actual set of properties used (from items 2-3 above) by using > the "debug" command (Drill 1.7 or later): > $ drillbit.sh debug -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-5046) Add documentation for directory based partition pruning
[ https://issues.apache.org/jira/browse/DRILL-5046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-5046: Fix Version/s: (was: 1.15.0) 1.16.0 > Add documentation for directory based partition pruning > --- > > Key: DRILL-5046 > URL: https://issues.apache.org/jira/browse/DRILL-5046 > Project: Apache Drill > Issue Type: Bug > Components: Documentation >Affects Versions: 1.9.0 >Reporter: Rahul Challapalli >Assignee: Bridget Bevens >Priority: Major > Fix For: 1.16.0 > > > Drill's documentation for partition pruning should cover the below 2 features > 1. Directory based partition pruning > 2. Partition pruning based on auto-partitioned parquet files > The first one seems to be missing from our documentation. At the very least > we should cover > a. How we can leverage this feature to avoid full table scans > b. How this feature works in-conjunction with metadata cache pruning > c. A few examples which involve using wildcards for one of the sub-directories -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703436#comment-16703436
]
ASF GitHub Bot commented on DRILL-6863:
---
vdiravka commented on a change in pull request #1557: DRILL-6863: Drop table is
not working if path within workspace starts…
URL: https://github.com/apache/drill/pull/1557#discussion_r237551997
##
File path: exec/java-exec/src/test/java/org/apache/drill/TestDropTable.java
##
@@ -21,23 +21,28 @@
import org.apache.drill.categories.UnlikelyTest;
import org.apache.drill.common.exceptions.UserException;
import org.apache.hadoop.fs.Path;
+import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.Assert;
import org.junit.experimental.categories.Category;
@Category(SqlTest.class)
public class TestDropTable extends PlanTestBase {
- private static final String CREATE_SIMPLE_TABLE = "create table %s as select
1 from cp.`employee.json`";
- private static final String CREATE_SIMPLE_VIEW = "create view %s as select 1
from cp.`employee.json`";
- private static final String DROP_TABLE = "drop table %s";
- private static final String DROP_TABLE_IF_EXISTS = "drop table if exists %s";
- private static final String DROP_VIEW_IF_EXISTS = "drop view if exists %s";
- private static final String BACK_TICK = "`";
+ private static final String CREATE_SIMPLE_TABLE = "create table `%s` as
select 1 from cp.`employee.json`";
Review comment:
I have not noticed it :) Thank you!
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.15.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6666) Doc link to AOL data set is broken
[ https://issues.apache.org/jira/browse/DRILL-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-: Fix Version/s: (was: 1.15.0) 1.16.0 > Doc link to AOL data set is broken > -- > > Key: DRILL- > URL: https://issues.apache.org/jira/browse/DRILL- > Project: Apache Drill > Issue Type: Bug > Components: Documentation >Affects Versions: 1.13.0 >Reporter: Paul Rogers >Assignee: Bridget Bevens >Priority: Minor > Fix For: 1.16.0 > > > Drill provides links to sample data sets in the documentation. Look in the > side bar, under Sample Datasets. Click on the "AOL Search" link. You'll get a > 404 error. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6728) DRILL-4864 - doc udfs for date, time, timestamp functions
[ https://issues.apache.org/jira/browse/DRILL-6728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6728: Fix Version/s: (was: 1.15.0) 1.16.0 > DRILL-4864 - doc udfs for date, time, timestamp functions > - > > Key: DRILL-6728 > URL: https://issues.apache.org/jira/browse/DRILL-6728 > Project: Apache Drill > Issue Type: Improvement > Components: Documentation >Reporter: Bridget Bevens >Assignee: Bridget Bevens >Priority: Major > Fix For: 1.16.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6787) Update Spnego webpage
[
https://issues.apache.org/jira/browse/DRILL-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6787:
Fix Version/s: (was: 1.15.0)
1.16.0
> Update Spnego webpage
> -
>
> Key: DRILL-6787
> URL: https://issues.apache.org/jira/browse/DRILL-6787
> Project: Apache Drill
> Issue Type: Bug
> Components: Documentation
>Affects Versions: 1.14.0
>Reporter: Robert Hou
>Assignee: Bridget Bevens
>Priority: Major
> Labels: doc-impacting
> Fix For: 1.16.0
>
>
> A few things should be updated on this webpage:
> https://drill.apache.org/docs/configuring-drill-to-use-spnego-for-http-authentication/
> When configuring drillbits in drill-override.conf, the principal and keytab
> should be corrected. There are two places where this should be corrected.
> {noformat}
> drill.exec.http: {
> auth.spnego.principal:"HTTP/hostname@realm",
> auth.spnego.keytab:"path/to/keytab",
> auth.mechanisms: [“SPNEGO”]
> }
> {noformat}
> For the section on Chrome, we should change "hostname/domain" to "domain".
> Also, the two blanks around the "=" should be removed.
> {noformat}
> google-chrome --auth-server-whitelist="domain"
> example: google-chrome --auth-server-whitelist="machine.example.com"
> example: google-chrome --auth-server-whitelist="*.example.com"
> The IP address can also be used
> example: google-chrome --auth-server-whitelist="10.10.100.101"
> The URL given to Chrome to access the Web UI should match the domain
> specified in auth-server-whitelist. If the domain is used in
> auth-server-whitelist, then the domain should be used with Chrome. If the IP
> address is used in auth-server-whitelist, then the IP address should be used
> with Chrome.
> {noformat}
> Also, Linux and Mac should be treated in separate paragraphs. These should
> be the directions for Mac:
> {noformat}
> cd /Applications/Google Chrome.app/Contents/MacOS
> ./"Google Chrome" --auth-server-whitelist="*.example.com"
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6794) Document the JDBC properties required to retrieve result sets in batches while querying large tables
[ https://issues.apache.org/jira/browse/DRILL-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6794: Fix Version/s: (was: 1.15.0) 1.16.0 > Document the JDBC properties required to retrieve result sets in batches > while querying large tables > > > Key: DRILL-6794 > URL: https://issues.apache.org/jira/browse/DRILL-6794 > Project: Apache Drill > Issue Type: Bug > Components: Documentation >Affects Versions: 1.14.0 >Reporter: Rahul Raj >Assignee: Bridget Bevens >Priority: Major > Labels: doc-impacting > Fix For: 1.16.0 > > > Document the JDBC properties required to retrieve result sets in batches > while querying large tables > Querying large tables using JDBC plugin causes OOM as most JDBC drivers cache > the entire result set at the client by default. > To avoid this additional parameters needs to be specified with the JDBC > connection string so that the driver fetches records in batches and reloads > when exhausted. > For postgres driver set autocommit mode to false - > jdbc:postgresql://url:port/schema?defaultAutoCommit=false > Links > [1] https://issues.apache.org/jira/browse/DRILL-4177 > [2] https://jdbc.postgresql.org/documentation/93/query.html#fetchsize-example > [3] https://www.postgresql.org/docs/9.3/static/ecpg-sql-set-autocommit.html > [4] https://jdbc.postgresql.org/documentation/head/ds-cpds.htm -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6582) SYSLOG (RFC-5424) Format Plugin
[ https://issues.apache.org/jira/browse/DRILL-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6582: Fix Version/s: (was: 1.15.0) 1.16.0 > SYSLOG (RFC-5424) Format Plugin > --- > > Key: DRILL-6582 > URL: https://issues.apache.org/jira/browse/DRILL-6582 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: Future >Reporter: Charles Givre >Assignee: Charles Givre >Priority: Major > Labels: doc-impacting > Fix For: 1.16.0 > > > Many security log files are in the format defined by RFC-5424. A format > plugin which can read data formatted in according to this specification would > be very useful for security engineers as well as network engineers. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6791) Merge scan projection framework into master
[ https://issues.apache.org/jira/browse/DRILL-6791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6791: Fix Version/s: (was: 1.15.0) 1.16.0 > Merge scan projection framework into master > --- > > Key: DRILL-6791 > URL: https://issues.apache.org/jira/browse/DRILL-6791 > Project: Apache Drill > Issue Type: Improvement >Affects Versions: 1.15.0 >Reporter: Paul Rogers >Assignee: Paul Rogers >Priority: Major > Fix For: 1.16.0 > > > Merge the next set of "result set loader" code into master via a PR. This one > covers the "schema projection" mechanism which: > * Handles none (SELECT COUNT\(*)), some (SELECT a, b, x) and all (SELECT *) > projection. > * Handles null columns (for projection a column "x" that does not exist in > the base table.) > * Handles constant columns as used for file metadata (AKA "implicit" columns). > * Handle schema persistence: the need to reuse the same vectors across > different scanners > * Provides a framework for consuming externally-supplied metadata > * Since we don't yet have a way to provide "real" metadata, obtains metadata > hints from previous batches and from the projection list (a.b implies that > "a" is a map, c[0] implies that "c" is an array, etc.) > * Handles merging the set of data source columns and null columns to create > the final output batch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6863:
Affects Version/s: (was: 1.15.0)
1.14.0
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.15.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6562) Plugin Management improvements
[ https://issues.apache.org/jira/browse/DRILL-6562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vitalii Diravka updated DRILL-6562: --- Fix Version/s: (was: 1.15.0) 1.16.0 > Plugin Management improvements > -- > > Key: DRILL-6562 > URL: https://issues.apache.org/jira/browse/DRILL-6562 > Project: Apache Drill > Issue Type: Improvement > Components: Client - HTTP, Web Server >Affects Versions: 1.14.0 >Reporter: Abhishek Girish >Assignee: Vitalii Diravka >Priority: Major > Labels: doc-impacting > Fix For: 1.16.0 > > Attachments: image-2018-07-23-02-55-02-024.png, > image-2018-10-22-20-20-24-658.png, image-2018-10-22-20-20-59-105.png > > > Follow-up to DRILL-4580. > Provide ability to export all storage plugin configurations at once, with a > new "Export All" option on the Storage page of the Drill web UI -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703435#comment-16703435
]
ASF GitHub Bot commented on DRILL-6863:
---
vdiravka commented on a change in pull request #1557: DRILL-6863: Drop table is
not working if path within workspace starts…
URL: https://github.com/apache/drill/pull/1557#discussion_r237552734
##
File path:
exec/java-exec/src/test/java/org/apache/drill/exec/sql/TestViewSupport.java
##
@@ -751,4 +751,53 @@ public void selectFromViewCreatedOnCalcite1_4() throws
Exception {
.baselineValues("HeadQuarters")
.go();
}
+
+ @Test
+ public void testDropViewNameStartsWithSlash() throws Exception {
+String viewName = "views/tmp_view";
+try {
+ test("CREATE VIEW `%s`.`%s` AS SELECT * FROM cp.`region.json`",
DFS_TMP_SCHEMA, viewName);
+ testBuilder()
+ .sqlQuery("DROP VIEW `%s`.`%s`", DFS_TMP_SCHEMA, "/" + viewName)
+ .unOrdered()
+ .baselineColumns("ok", "summary")
+ .baselineValues(true,
+ String.format("View [%s] deleted successfully from schema
[%s].", viewName, DFS_TMP_SCHEMA))
+ .go();
+} finally {
+ test("DROP VIEW IF EXISTS `%s`.`%s`", DFS_TMP_SCHEMA, viewName);
+}
+ }
+
+ @Test
+ public void testViewIsCreatedWithinWorkspace() throws Exception {
+String viewName = "views/tmp_view";
+try {
+ test("CREATE VIEW `%s`.`%s` AS SELECT * FROM cp.`region.json`",
DFS_TMP_SCHEMA, "/" + viewName);
+ testBuilder()
+ .sqlQuery("SELECT region_id FROM `%s`.`%s` LIMIT 1", DFS_TMP_SCHEMA,
viewName)
+ .unOrdered()
+ .baselineColumns("region_id")
+ .baselineValues(0L)
+ .go();
+} finally {
+ test("DROP VIEW IF EXISTS `%s`.`%s`", DFS_TMP_SCHEMA, viewName);
+}
+ }
+
+ @Test
+ public void testViewIsFoundWithinWorkspaceWhenNameStartsWithSlash() throws
Exception {
Review comment:
It's fine
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.15.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitalii Diravka updated DRILL-6863:
---
Fix Version/s: (was: 1.16.0)
1.15.0
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.15.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vitalii Diravka updated DRILL-6863:
---
Labels: ready-to-commit (was: )
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Labels: ready-to-commit
> Fix For: 1.15.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703293#comment-16703293
]
ASF GitHub Bot commented on DRILL-6863:
---
arina-ielchiieva commented on a change in pull request #1557: DRILL-6863: Drop
table is not working if path within workspace starts…
URL: https://github.com/apache/drill/pull/1557#discussion_r237516001
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DropTableHandler.java
##
@@ -56,7 +57,7 @@ public DropTableHandler(SqlHandlerConfig config) {
@Override
public PhysicalPlan getPlan(SqlNode sqlNode) throws ValidationException,
RelConversionException, IOException {
SqlDropTable dropTableNode = ((SqlDropTable) sqlNode);
-String originalTableName = dropTableNode.getName();
+String originalTableName =
FileSelection.removeLeadingSlash(dropTableNode.getName());
Review comment:
Sounds fair, thanks for the explanation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.16.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703287#comment-16703287
]
ASF GitHub Bot commented on DRILL-6863:
---
KazydubB commented on a change in pull request #1557: DRILL-6863: Drop table is
not working if path within workspace starts…
URL: https://github.com/apache/drill/pull/1557#discussion_r237475983
##
File path: exec/java-exec/src/test/java/org/apache/drill/TestDropTable.java
##
@@ -220,4 +218,36 @@ public void testDropTableIfExistsWhileItIsAView() throws
Exception {
test(DROP_VIEW_IF_EXISTS, viewName);
}
}
+
+ @Test
+ public void testDropTableNameStartsWithSlash() throws Exception {
+String tableName = "test_table";
+try {
+ test(CREATE_SIMPLE_TABLE, tableName);
+ testBuilder()
+ .sqlQuery(DROP_TABLE, "/" + tableName)
+ .unOrdered()
+ .baselineColumns("ok", "summary")
+ .baselineValues(true, String.format("Table [%s] dropped", tableName))
+ .go();
+} finally {
+ test(DROP_TABLE_IF_EXISTS, tableName);
+}
+ }
+
+ @Test
+ public void testDropTemporaryTableNameStartsWithSlash() throws Exception {
Review comment:
This case is (implicitly) covered in
`TestCTTAS#testTemporaryTableWithAndWithoutLeadingSlashAreTheSame()` as it
ensures that table w/ and w/o slash are the same during creation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.16.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703286#comment-16703286
]
ASF GitHub Bot commented on DRILL-6863:
---
KazydubB commented on a change in pull request #1557: DRILL-6863: Drop table is
not working if path within workspace starts…
URL: https://github.com/apache/drill/pull/1557#discussion_r237506645
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DropTableHandler.java
##
@@ -56,7 +57,7 @@ public DropTableHandler(SqlHandlerConfig config) {
@Override
public PhysicalPlan getPlan(SqlNode sqlNode) throws ValidationException,
RelConversionException, IOException {
SqlDropTable dropTableNode = ((SqlDropTable) sqlNode);
-String originalTableName = dropTableNode.getName();
+String originalTableName =
FileSelection.removeLeadingSlash(dropTableNode.getName());
Review comment:
Actually, this may appear counter-intuitive for users especially when Drill
supports `SELECT` with leading slash (see
[DRILL-1130](https://issues.apache.org/jira/browse/DRILL-1130)). When a table
is created and its name does start with '/' it will be created within
workspace, i.e. will be rooted from the workspace.
You could think of it as if you performed `touch` (on Unix[-like] OS) to
create a file: `touch /${filename}` and `touch ${filename}` should create a
file in the same directory when working directory (workspace) is root ('/').
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.16.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6875) Drill doesn't try to update connection for S3 after session expired
[
https://issues.apache.org/jira/browse/DRILL-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Ordynskiy updated DRILL-6875:
---
Priority: Major (was: Minor)
> Drill doesn't try to update connection for S3 after session expired
> ---
>
> Key: DRILL-6875
> URL: https://issues.apache.org/jira/browse/DRILL-6875
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.16.0
>
> Attachments: drillbit.log
>
>
> *Steps to reproduce:*
> - Drill has S3 storage plugin.
> - Open sqlline and run query to S3.
> - Leave sqlline opened for more than 12 hours.
> - In opened sqlline run query to S3.
> *Expected result:*
> Drill should update authorization session and successfully execute query.
> *Actual result:*
> Sqlline returns an error:
> *{color:#d04437}Error: VALIDATION ERROR: Forbidden (Service: Amazon S3;
> Status Code: 403; Error Code: 403 Forbidden; Request ID: 4A94DD331A035625; S3
> Extended Request ID:
> uy94YdRpQ3ZriCz9xbnDi0yinB4O9kGrH7XPAURhjh8WZoxsbawojQA6v7mfvu920yOYbEI5WP8=)
> [Error Id: 4b44a83b-0e47-45a4-92e3-75f94f5a70cb on maprhost:31010]
> (state=,code=0){color}*
> *Reopening sqlline doesn't help to get S3 access.*
> *Access problem can be solved only by restarting Drill.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6875) Drill doesn't try to update connection for S3 after session expired
[
https://issues.apache.org/jira/browse/DRILL-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6875:
Fix Version/s: 1.16.0
> Drill doesn't try to update connection for S3 after session expired
> ---
>
> Key: DRILL-6875
> URL: https://issues.apache.org/jira/browse/DRILL-6875
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Minor
> Fix For: 1.16.0
>
> Attachments: drillbit.log
>
>
> *Steps to reproduce:*
> - Drill has S3 storage plugin.
> - Open sqlline and run query to S3.
> - Leave sqlline opened for more than 12 hours.
> - In opened sqlline run query to S3.
> *Expected result:*
> Drill should update authorization session and successfully execute query.
> *Actual result:*
> Sqlline returns an error:
> *{color:#d04437}Error: VALIDATION ERROR: Forbidden (Service: Amazon S3;
> Status Code: 403; Error Code: 403 Forbidden; Request ID: 4A94DD331A035625; S3
> Extended Request ID:
> uy94YdRpQ3ZriCz9xbnDi0yinB4O9kGrH7XPAURhjh8WZoxsbawojQA6v7mfvu920yOYbEI5WP8=)
> [Error Id: 4b44a83b-0e47-45a4-92e3-75f94f5a70cb on maprhost:31010]
> (state=,code=0){color}*
> *Reopening sqlline doesn't help to get S3 access.*
> *Access problem can be solved only by restarting Drill.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (DRILL-6875) Drill doesn't try to update connection for S3 after session expired
[
https://issues.apache.org/jira/browse/DRILL-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva reassigned DRILL-6875:
---
Assignee: Bohdan Kazydub
> Drill doesn't try to update connection for S3 after session expired
> ---
>
> Key: DRILL-6875
> URL: https://issues.apache.org/jira/browse/DRILL-6875
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Minor
> Attachments: drillbit.log
>
>
> *Steps to reproduce:*
> - Drill has S3 storage plugin.
> - Open sqlline and run query to S3.
> - Leave sqlline opened for more than 12 hours.
> - In opened sqlline run query to S3.
> *Expected result:*
> Drill should update authorization session and successfully execute query.
> *Actual result:*
> Sqlline returns an error:
> *{color:#d04437}Error: VALIDATION ERROR: Forbidden (Service: Amazon S3;
> Status Code: 403; Error Code: 403 Forbidden; Request ID: 4A94DD331A035625; S3
> Extended Request ID:
> uy94YdRpQ3ZriCz9xbnDi0yinB4O9kGrH7XPAURhjh8WZoxsbawojQA6v7mfvu920yOYbEI5WP8=)
> [Error Id: 4b44a83b-0e47-45a4-92e3-75f94f5a70cb on maprhost:31010]
> (state=,code=0){color}*
> *Reopening sqlline doesn't help to get S3 access.*
> *Access problem can be solved only by restarting Drill.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6875) Drill doesn't try to update connection for S3 after session expired
[
https://issues.apache.org/jira/browse/DRILL-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arina Ielchiieva updated DRILL-6875:
Affects Version/s: (was: 1.15.0)
1.14.0
> Drill doesn't try to update connection for S3 after session expired
> ---
>
> Key: DRILL-6875
> URL: https://issues.apache.org/jira/browse/DRILL-6875
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.14.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Minor
> Attachments: drillbit.log
>
>
> *Steps to reproduce:*
> - Drill has S3 storage plugin.
> - Open sqlline and run query to S3.
> - Leave sqlline opened for more than 12 hours.
> - In opened sqlline run query to S3.
> *Expected result:*
> Drill should update authorization session and successfully execute query.
> *Actual result:*
> Sqlline returns an error:
> *{color:#d04437}Error: VALIDATION ERROR: Forbidden (Service: Amazon S3;
> Status Code: 403; Error Code: 403 Forbidden; Request ID: 4A94DD331A035625; S3
> Extended Request ID:
> uy94YdRpQ3ZriCz9xbnDi0yinB4O9kGrH7XPAURhjh8WZoxsbawojQA6v7mfvu920yOYbEI5WP8=)
> [Error Id: 4b44a83b-0e47-45a4-92e3-75f94f5a70cb on maprhost:31010]
> (state=,code=0){color}*
> *Reopening sqlline doesn't help to get S3 access.*
> *Access problem can be solved only by restarting Drill.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (DRILL-6875) Drill doesn't try to update connection for S3 after session expired
Denys Ordynskiy created DRILL-6875:
--
Summary: Drill doesn't try to update connection for S3 after
session expired
Key: DRILL-6875
URL: https://issues.apache.org/jira/browse/DRILL-6875
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.15.0
Reporter: Denys Ordynskiy
Attachments: drillbit.log
*Steps to reproduce:*
- Drill has S3 storage plugin.
- Open sqlline and run query to S3.
- Leave sqlline opened for more than 12 hours.
- In opened sqlline run query to S3.
*Expected result:*
Drill should update authorization session and successfully execute query.
*Actual result:*
Sqlline returns an error:
*{color:#d04437}Error: VALIDATION ERROR: Forbidden (Service: Amazon S3; Status
Code: 403; Error Code: 403 Forbidden; Request ID: 4A94DD331A035625; S3 Extended
Request ID:
uy94YdRpQ3ZriCz9xbnDi0yinB4O9kGrH7XPAURhjh8WZoxsbawojQA6v7mfvu920yOYbEI5WP8=)
[Error Id: 4b44a83b-0e47-45a4-92e3-75f94f5a70cb on maprhost:31010]
(state=,code=0){color}*
*Reopening sqlline doesn't help to get S3 access.*
*Access problem can be solved only by restarting Drill.*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703090#comment-16703090
]
ASF GitHub Bot commented on DRILL-6863:
---
arina-ielchiieva commented on a change in pull request #1557: DRILL-6863: Drop
table is not working if path within workspace starts…
URL: https://github.com/apache/drill/pull/1557#discussion_r237463189
##
File path: exec/java-exec/src/test/java/org/apache/drill/TestDropTable.java
##
@@ -220,4 +218,36 @@ public void testDropTableIfExistsWhileItIsAView() throws
Exception {
test(DROP_VIEW_IF_EXISTS, viewName);
}
}
+
+ @Test
+ public void testDropTableNameStartsWithSlash() throws Exception {
Review comment:
Please move tests for CTTAS to the TestCTTAS class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.16.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703092#comment-16703092
]
ASF GitHub Bot commented on DRILL-6863:
---
arina-ielchiieva commented on a change in pull request #1557: DRILL-6863: Drop
table is not working if path within workspace starts…
URL: https://github.com/apache/drill/pull/1557#discussion_r237463961
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DropTableHandler.java
##
@@ -56,7 +57,7 @@ public DropTableHandler(SqlHandlerConfig config) {
@Override
public PhysicalPlan getPlan(SqlNode sqlNode) throws ValidationException,
RelConversionException, IOException {
SqlDropTable dropTableNode = ((SqlDropTable) sqlNode);
-String originalTableName = dropTableNode.getName();
+String originalTableName =
FileSelection.removeLeadingSlash(dropTableNode.getName());
Review comment:
Maybe instead of removing slashes we should disallow creating such tables?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.16.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703094#comment-16703094
]
ASF GitHub Bot commented on DRILL-6863:
---
arina-ielchiieva commented on a change in pull request #1557: DRILL-6863: Drop
table is not working if path within workspace starts…
URL: https://github.com/apache/drill/pull/1557#discussion_r237463961
##
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DropTableHandler.java
##
@@ -56,7 +57,7 @@ public DropTableHandler(SqlHandlerConfig config) {
@Override
public PhysicalPlan getPlan(SqlNode sqlNode) throws ValidationException,
RelConversionException, IOException {
SqlDropTable dropTableNode = ((SqlDropTable) sqlNode);
-String originalTableName = dropTableNode.getName();
+String originalTableName =
FileSelection.removeLeadingSlash(dropTableNode.getName());
Review comment:
Maybe instead of removing slashes we should disallow creating such tables?
Let's say if we detect that table starts with slash fail and report any error
and table naming recommendations.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.16.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703091#comment-16703091
]
ASF GitHub Bot commented on DRILL-6863:
---
arina-ielchiieva commented on a change in pull request #1557: DRILL-6863: Drop
table is not working if path within workspace starts…
URL: https://github.com/apache/drill/pull/1557#discussion_r237463405
##
File path: exec/java-exec/src/test/java/org/apache/drill/TestDropTable.java
##
@@ -220,4 +218,36 @@ public void testDropTableIfExistsWhileItIsAView() throws
Exception {
test(DROP_VIEW_IF_EXISTS, viewName);
}
}
+
+ @Test
+ public void testDropTableNameStartsWithSlash() throws Exception {
+String tableName = "test_table";
+try {
+ test(CREATE_SIMPLE_TABLE, tableName);
+ testBuilder()
+ .sqlQuery(DROP_TABLE, "/" + tableName)
+ .unOrdered()
+ .baselineColumns("ok", "summary")
+ .baselineValues(true, String.format("Table [%s] dropped", tableName))
+ .go();
+} finally {
+ test(DROP_TABLE_IF_EXISTS, tableName);
+}
+ }
+
+ @Test
+ public void testDropTemporaryTableNameStartsWithSlash() throws Exception {
Review comment:
What if user has created table with slash initially?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.16.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703089#comment-16703089
]
ASF GitHub Bot commented on DRILL-6863:
---
arina-ielchiieva commented on a change in pull request #1557: DRILL-6863: Drop
table is not working if path within workspace starts…
URL: https://github.com/apache/drill/pull/1557#discussion_r237463189
##
File path: exec/java-exec/src/test/java/org/apache/drill/TestDropTable.java
##
@@ -220,4 +218,36 @@ public void testDropTableIfExistsWhileItIsAView() throws
Exception {
test(DROP_VIEW_IF_EXISTS, viewName);
}
}
+
+ @Test
+ public void testDropTableNameStartsWithSlash() throws Exception {
Review comment:
Please move test for CTTAS to the TestCTTAS class.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.16.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6769) Sqlline doesn't see line endings when it prints first row fo the explain plan results table
[ https://issues.apache.org/jira/browse/DRILL-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arina Ielchiieva updated DRILL-6769: Reviewer: Volodymyr Vysotskyi > Sqlline doesn't see line endings when it prints first row fo the explain plan > results table > --- > > Key: DRILL-6769 > URL: https://issues.apache.org/jira/browse/DRILL-6769 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.15.0 >Reporter: Denys Ordynskiy >Assignee: Arina Ielchiieva >Priority: Major > Fix For: 1.15.0 > > Attachments: Sqlline.png > > > After Sqlline was upgraded in DRILL-3853, in the query "EXPLAIN PLAN FOR .." > results table has a very long first row with titles. > After I run the query "EXPLAIN PLAN FOR SELECT 1;", width of the table column > is calculated by the number of symbols in the biggest cell of columns. > If there are new lines in the data, Sqlline should set width of this column > by the longest line of the cells data instead of number of all symbols of > that cell. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (DRILL-6769) Sqlline doesn't see line endings when it prints first row fo the explain plan results table
[ https://issues.apache.org/jira/browse/DRILL-6769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16701751#comment-16701751 ] Arina Ielchiieva edited comment on DRILL-6769 at 11/29/18 11:38 AM: maxColumndWidth will be set to 80. was (Author: arina): maxColumndWidth will be set to 100. > Sqlline doesn't see line endings when it prints first row fo the explain plan > results table > --- > > Key: DRILL-6769 > URL: https://issues.apache.org/jira/browse/DRILL-6769 > Project: Apache Drill > Issue Type: Bug >Affects Versions: 1.15.0 >Reporter: Denys Ordynskiy >Assignee: Arina Ielchiieva >Priority: Major > Fix For: 1.15.0 > > Attachments: Sqlline.png > > > After Sqlline was upgraded in DRILL-3853, in the query "EXPLAIN PLAN FOR .." > results table has a very long first row with titles. > After I run the query "EXPLAIN PLAN FOR SELECT 1;", width of the table column > is calculated by the number of symbols in the biggest cell of columns. > If there are new lines in the data, Sqlline should set width of this column > by the longest line of the cells data instead of number of all symbols of > that cell. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703051#comment-16703051
]
ASF GitHub Bot commented on DRILL-6863:
---
KazydubB commented on a change in pull request #1557: DRILL-6863: Drop table is
not working if path within workspace starts…
URL: https://github.com/apache/drill/pull/1557#discussion_r237396700
##
File path:
exec/java-exec/src/test/java/org/apache/drill/exec/sql/TestViewSupport.java
##
@@ -751,4 +751,53 @@ public void selectFromViewCreatedOnCalcite1_4() throws
Exception {
.baselineValues("HeadQuarters")
.go();
}
+
+ @Test
+ public void testDropViewNameStartsWithSlash() throws Exception {
+String viewName = "views/tmp_view";
+try {
+ test("CREATE VIEW `%s`.`%s` AS SELECT * FROM cp.`region.json`",
DFS_TMP_SCHEMA, viewName);
+ testBuilder()
+ .sqlQuery("DROP VIEW `%s`.`%s`", DFS_TMP_SCHEMA, "/" + viewName)
+ .unOrdered()
+ .baselineColumns("ok", "summary")
+ .baselineValues(true,
+ String.format("View [%s] deleted successfully from schema
[%s].", viewName, DFS_TMP_SCHEMA))
+ .go();
+} finally {
+ test("DROP VIEW IF EXISTS `%s`.`%s`", DFS_TMP_SCHEMA, viewName);
+}
+ }
+
+ @Test
+ public void testViewIsCreatedWithinWorkspace() throws Exception {
Review comment:
Added such test cases for tables.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.16.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703053#comment-16703053
]
ASF GitHub Bot commented on DRILL-6863:
---
KazydubB commented on a change in pull request #1557: DRILL-6863: Drop table is
not working if path within workspace starts…
URL: https://github.com/apache/drill/pull/1557#discussion_r237197837
##
File path:
exec/java-exec/src/test/java/org/apache/drill/exec/sql/TestViewSupport.java
##
@@ -751,4 +751,53 @@ public void selectFromViewCreatedOnCalcite1_4() throws
Exception {
.baselineValues("HeadQuarters")
.go();
}
+
+ @Test
+ public void testDropViewNameStartsWithSlash() throws Exception {
+String viewName = "views/tmp_view";
+try {
+ test("CREATE VIEW `%s`.`%s` AS SELECT * FROM cp.`region.json`",
DFS_TMP_SCHEMA, viewName);
+ testBuilder()
+ .sqlQuery("DROP VIEW `%s`.`%s`", DFS_TMP_SCHEMA, "/" + viewName)
+ .unOrdered()
+ .baselineColumns("ok", "summary")
+ .baselineValues(true,
+ String.format("View [%s] deleted successfully from schema
[%s].", viewName, DFS_TMP_SCHEMA))
+ .go();
+} finally {
+ test("DROP VIEW IF EXISTS `%s`.`%s`", DFS_TMP_SCHEMA, viewName);
+}
+ }
+
+ @Test
+ public void testViewIsCreatedWithinWorkspace() throws Exception {
+String viewName = "views/tmp_view";
+try {
+ test("CREATE VIEW `%s`.`%s` AS SELECT * FROM cp.`region.json`",
DFS_TMP_SCHEMA, "/" + viewName);
+ testBuilder()
+ .sqlQuery("SELECT region_id FROM `%s`.`%s` LIMIT 1", DFS_TMP_SCHEMA,
viewName)
+ .unOrdered()
+ .baselineColumns("region_id")
+ .baselineValues(0L)
+ .go();
+} finally {
+ test("DROP VIEW IF EXISTS `%s`.`%s`", DFS_TMP_SCHEMA, viewName);
+}
+ }
+
+ @Test
+ public void testViewIsFoundWithinWorkspaceWhenNameStartsWithSlash() throws
Exception {
Review comment:
You're right that this holds for default workspace. `WithinWorkspace` is
included to emphasize that the view is being searched for in a specified
workspace as opposed to being searched in `/views/tmp_view` (i.e. starting from
FS's root).
Should I omit it?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.16.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (DRILL-6863) Drop table is not working if path within workspace starts with '/'
[
https://issues.apache.org/jira/browse/DRILL-6863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16703052#comment-16703052
]
ASF GitHub Bot commented on DRILL-6863:
---
KazydubB commented on a change in pull request #1557: DRILL-6863: Drop table is
not working if path within workspace starts…
URL: https://github.com/apache/drill/pull/1557#discussion_r237193678
##
File path: exec/java-exec/src/test/java/org/apache/drill/TestDropTable.java
##
@@ -21,23 +21,28 @@
import org.apache.drill.categories.UnlikelyTest;
import org.apache.drill.common.exceptions.UserException;
import org.apache.hadoop.fs.Path;
+import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.Assert;
import org.junit.experimental.categories.Category;
@Category(SqlTest.class)
public class TestDropTable extends PlanTestBase {
- private static final String CREATE_SIMPLE_TABLE = "create table %s as select
1 from cp.`employee.json`";
- private static final String CREATE_SIMPLE_VIEW = "create view %s as select 1
from cp.`employee.json`";
- private static final String DROP_TABLE = "drop table %s";
- private static final String DROP_TABLE_IF_EXISTS = "drop table if exists %s";
- private static final String DROP_VIEW_IF_EXISTS = "drop view if exists %s";
- private static final String BACK_TICK = "`";
+ private static final String CREATE_SIMPLE_TABLE = "create table `%s` as
select 1 from cp.`employee.json`";
Review comment:
Surrounded `%s` with '`' (backtick) to allow table names containing '/'.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Drop table is not working if path within workspace starts with '/'
> --
>
> Key: DRILL-6863
> URL: https://issues.apache.org/jira/browse/DRILL-6863
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.15.0
>Reporter: Denys Ordynskiy
>Assignee: Bohdan Kazydub
>Priority: Major
> Fix For: 1.16.0
>
>
> Drill works incorrectly if path to the table within workspace starts with '/'
> Request "drop table s3.tmp.`drill/transitive_closure/tab1`" works fine,
> but if I add '/' in the begining of the tables path "drop table
> s3.tmp.`{color:#d04437}/{color}drill/transitive_closure/tab1`", Drill is
> trying to find table in the root directory but not in workspace path.
> *Actual result:*
> Drill returns successfully response
> "Table [/drill/transitive_closure/tab1] dropped"
> but table was not dropped.
>
> *Expected result:*
> Table was droped.
> Bug can be reproduced on S3 and DFS storages. On FileSystem storage Drill
> successfully returns error message if "drop table" query starts with '/' in
> table path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
